[<<][plt][>>][..]
Thu Nov 3 19:28:06 EDT 2011

still crashing

tom@zoo:~/plt$ time (/opt/apache-logs/sorted | racket ~/plt/lib/x/apachelog-dump.ss | tee /tmp/access.sql | sudo mysql -D apachelogs)
1145249 ERROR 1064 (42000) at line 2190300: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INTO entry (id_agent, id_date, id_req, id_ip, id_stat, id_ref) values (1650, 897' at line 2
1145308 error writing to stream port (Broken pipe; errno=32)

real    9m43.920s
user    8m43.313s
sys     1m26.421s
tom@zoo:~/plt$ 

Offending line:

INSERT INTO date (id_date, date) values (897508, #f);

It seems to be due to this one:


WARNING: can't handle date (4350104328545217515939634383792078420781148110824235358325269992011008006050 44 22 24 11 2010 3600)

Which I assume is this one, another botched line:

213.133.113.86 - - [24/Nov/2010:22:59:35 +0100] "GET / HTTP/1.1" 200 497 "-" "Hetzner System Monitorin213.133.113.84 - - [25/Nov/2010:03:05:53 +0100] "GET / HTTP/1.1" 200 497 "-" "Hetzner System Monitoring"

Maybe i should start anchoring lines and throw out matches that don't work that way..

It runs to completion.

$ time (/opt/apache-logs/sorted | racket ~/plt/lib/x/apachelog-dump.ss | tee /tmp/access.sql | sudo mysql -D apachelogs)
1145249 WARNING: can't handle date (4350104328545217515939634383792078420781148110824235358325269992011008006050 44 22 24 11 2010 3600)
4080617 ^[[C
real    49m46.360s
user    45m55.024s
sys     5m20.604s

Only 63981 unique IPs.  That's a surprise.

So I fixed up the date format to generate MySQL syntax.

That makes it a lot faster too!

$ time (/opt/apache-logs/sorted | racket ~/plt/lib/x/apachelog-dump.ss | tee /tmp/access.sql | sudo mysql -D apachelogs)
4080617 
real    11m7.764s
user    11m25.983s
sys     0m48.835s



[Reply][About]
[<<][plt][>>][..]