Select N and 1 Batch Insert

Select N and 1 Batch Insert

<p># of SELECT # of INSERTs Time ms 2 separate POSTs each 2 INSERTS in one POST 358 having one SELECT 1000 POSTs (1 SELECT 1000 in one POST 68698 per POST) 2000 POSTs (1 SELECT 2000 in one POST 137510 per POST)</p><p>1 POST with 100 - 278 SELECTs 1 POST with >120 - Blows up memory on SELECTs Server</p><p>Insert Performance</p><p>Select n and 1 batch insert</p><p># of POST request # of SELECT Clauses # of triples being looked Time (ms) up 1 1 100 278 1 1 >120 Gives memory error from server</p><p>“INSERT DATA” runtimes</p><p># of POSTs # of INSERT DATA # of triples inserted Time (ms) clauses 1 1 100 3226 146 152 172 1 1 1000 985 487 816 1 1 2000 927 771 875 1 1 3000 3913 1932 1001 1035 1 1 10000 Blows up memory on server</p><p>“INSERT WHERE” runtimes</p><p># of Posts # of INSERT WHERE Time (ms) Clauses 1 100 4353</p><p>1 1000 40863 1 2000 84105 1 3000 217595 1 10000 Blows up memory on Server</p><p>Note: for 10000 triple INSERT DATA update getting an error from server, error: memory exhausted at line 19986 of operation 0</p><p>>>>> N SELECT and M INSERT DATA runtimes:</p><p>Test 1] Linear 100 SELECT + 2000 INSERT DATA </p><p>Run1) 2409 ms</p><p>Run2) 2495ms </p><p>Run3) 2358ms</p><p>Test2] Linear SELECT + INSERT DATA Client = Laptop (from home network) 4Store Server = Losangeles VM</p><p>50000 (500 X 100) SELECT = 44859ms 50000 (25 X 2000) INSERT DATA = 43432ms Total time for above 2 operations = 88291ms Test3] Parallel SELECT + INSERT DATA test2 in Parallel with 5 threads</p><p>Client = Client = Laptop (from home network). All threads working on this machine 4Store Server = Losangeles VM</p><p>Time to do SELECT + INSERT DATA in Parallel = 65122 ms</p><p>Output from individual threads: Thread4 Done with Bulk SELECTs count = 10000 in 23175 ms Thread3 Done with Bulk SELECTs count = 10000 in 23305 ms Thread0 Done with Bulk SELECTs count = 10000 in 23329 ms Thread2 Done with Bulk SELECTs count = 10000 in 23369 ms Thread1 Done with Bulk SELECTs count = 10000 in 26994 ms</p><p>Thread4 Done ... Bulk INSERT DATA Response time = 59249ms Thread3 Done ... Bulk INSERT DATA Response time = 61482ms Thread0 Done ... Bulk INSERT DATA Response time = 62773ms Thread2 Done ... Bulk INSERT DATA Response time = 64024ms Thread1 Done ... Bulk INSERT DATA Response time = 65119ms</p><p>Time to do SELECT + INSERT DATA in Parallel = 65122 ms</p><p>Test4] Parallel INSERT DATA test2 in Parallel with 25 threads</p><p>Client = Laptop (from home network). All threads working on this machine 4Store Server = Losangeles VM</p><p>Time to do SELECT + INSERT DATA in Parallel = 68425 ms</p><p>Test5] Parallel (test 2 modified … increased number of INSERT DATA) # of Threads = 5</p><p>Client = Laptop (from home network) 4Store Server = Losangeles VM</p><p>50000 SELECT AND (30X2000) 60000 INSERT DATA</p><p>Time to do SELECT + INSERT DATA in Parallel = 72734 ms Test6] Parallel (test 2 modified … increased number of INSERT DATA) # of Threads = 10</p><p>Client = Laptop (from home network) 4Store Server = Losangeles VM</p><p>50000 SELECT AND (30X2000) 60000 INSERT DATA</p><p>Time to do SELECT + INSERT DATA in Parallel = 77111 ms</p><p>Test7] Parallel (test 2 modified … increased number of INSERT DATA) # of Threads = 15</p><p>Client = Laptop (from home network) 4Store Server = Losangeles VM</p><p>50000 SELECT AND (30X2000) 60000 INSERT DATA</p><p>Time to do SELECT + INSERT DATA in Parallel = 76943 ms</p><p>Summary:</p><p>Test # Parallel / # of threads SELECT INSER DATA Time (ms) Linear Triples Triples 1 Linear 1 100 2000 2409 2 Linear 1 50000 50000 88291 3 Parallel 5 50000 50000 65122 4 Parallel 25 50000 50000 68425 5 Parallel 5 50000 60000 72734 6 Parallel 10 50000 60000 77111</p>

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    4 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us