This is a public chat log generated from the #semsol IRC channel.
14:57:49
Hi, I have a question about arc load
15:01:43
just ask :)
15:03:30
thanks
15:03:57
the thing is that I want to upload a big rdf file into the mysql-based store
15:04:19
but there ir no way I can do it using the sparql+ endpoint
15:04:47
so I was wondering if there is another way to do it or how to "translate" rdf tripls to sql inserts for the given schema
15:05:25
LOAD <path/to/file> doesn't work?
15:06:58
it does work BUT there isn't enough memory to load the while graph before loading it
15:07:29
ARC should stream-load with constant mem
15:07:51
I can't do such a big use memory and I don't have control over the php configuration
15:08:14
do any triples end up in the mysql tables at all?
15:08:29
you can reduce the insert batch size
15:09:18
.. by specifying a low "store_write_buffer" in the configuration
15:09:27
default is 2500
15:09:56
that's the number of triples that are buffered in memory before they are sent to mysql
15:10:42
yes, there are some inserts but not so much
15:10:48
I will try again
15:11:05
the batch config is available from revision 2009-03-04
15:11:56
I've been able to load 3MT with PHP memory set to 8MB
15:11:58
ok, I'll check that
15:12:09
is the input doc RDF/XML?
15:12:23
yes
15:12:26
ok
15:12:42
i was wondering if it would be better to convert it to ntriples
15:13:03
rdf/xml is safest, stream-wise
15:14:28
the ntriples/turtle parser sometimes doesn't find delimiters in non-trivial markup and then fills up the memory
15:14:50
ok, I see