Re: [arc-dev] MySQL errors
From: Alexandre Passant
Subject: Re: [arc-dev] MySQL errors
Date: Sun, 11 May 2008 18:42:50 +0200
Hi,
Thanks for your quick feedback and the union bugfix.
Unfortunately, I'm afraid I can't help regarding the memory bug :-/
Actually, I mailed dreamhost at the same time I reported the bug, as I =20=
was a bit surprised by this memory error that I never got before.
They noticed that a script on the same server was using too much =20
memory (making my ARC2 instance fail) and fixed it but now I'm unable =20=
to reproduce the memory error ...
If you want, I can setup ARC2 on a local machine and check the amount =20=
of memory used before and after the changes you suggest.
Talk soon,
Alex.
Le 8 mai 08 =E0 13:58, Benjamin Nowack a =E9crit :
>
>
> Hi Alex,
>
> I managed to fix the UNION syntax error, thanks a lot for the report.
> I couldn't reproduce the memory error, though, but found a couple of
> related reports in the mysql bug DB. It's somehow related to mysql
> incorrectly allocating too much memory. Given that this didn't happen
> with earlier ARC versions, I should be able to get rid of this =20
> somehow.
> One thing I added recently was the use of in-MEMORY intermediate =20
> tables
> for newer MySQL servers and SELECT queries. I also changed a couple of
> table indexes. So, maybe the MySQL server chokes on the MEMORY tables.
>
> Unfortunately, I'm in the middle of a major new rev (just aligned the
> internal structures with the Talis platform [1]), so I can't put up a
> new version yet. Could you perhaps try to tweak a line in the
> SelectQueryHandler to see if switching from MEMORY to MYISAM makes
> any difference? You can try two things: First, s/MEMORY/MYISAM/ in
> ARC2_StoreSelectQueryHandler.php, line 94:
>
> $tmp_sql .=3D ($v < '04-01-00') ? '=3DMYISAM' : '=3DMYISAM';
>
> I'm not sure if MySQL perhaps needs more mem for TEMPORARY tables, so
> you could additionally remove the TEMPORARY keyword in line 91:
>
> $tmp_sql =3D 'CREATE TABLE ' . $tmp_tbl . ' ( ' ...
>
> If these changes make the error go away, we've probably found the =20
> bug's
> origin and I can fix it in the next revision. It would be really =20
> great if
> you could have a try and help me, as I can't reproduce the memory
> bottleneck here.
>
> Thanks in advance,
> Benji
>
>
> [1]
> =
http://arc.semsol.org/community/arc-dev/archives/2008/01/PM-GA.20080131094=
910.4C
> 65A.2.1D@semsol.com
>
> --
> Benjamin Nowack
> http://bnode.org/
>
>
> On 07.05.2008 21:47:10, Alexandre Passant wrote:
>>
>>
>> On Wed, May 7, 2008 at 12:52 PM, Benjamin Nowack =20
>> <bnowack@semsol.com> wrote:
>>>
>>>
>>> Hi Alex,
>>> The new version uses a different approach when intermediate
>>> tables are created ("CREATE + INSERT SELECT" instead of a direct
>>> "CREATE SELECT"), but that shouldn't cause PHP-level memory
>>> issues. I did change some memory/buffer settings, but only for
>>> LOADs. Could you tell me how many triples your store contains,
>>> and also which concrete queries you used? Then I should (hopefully)
>>> be able to reproduce the bug here. (You can also send me a MySQL
>>> dump off-list, if you like, and if the data aren't too secret)
>>
>> The store is relatively small (I just re-created a new one with less
>> than 100 triples), and even a 'select ?s ?p ?o where {?s ?p ?o}' =20
>> query
>> returned the memory error.
>> Yet, I ran a few more tests, and actually, the SELECT queries I used
>> work randomly, so it seems they do not always consume the same amount
>> of memory.
>> Yet, I don't have any problem with the LOAD ones.
>>
>> I'll send you the dump and example queries off-list.
>>
>> Thanks,
>> Alex.
>>
>>>
>>> Benji
>>>
>>>
>>>
>>> On 07.05.2008 11:52:03, Alexandre Passant wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I've just updated some of my code to the latest version of ARC =20
>>>> and get
>>>> the following errors:
>>>>
>>>> When I run a simple query to get all the statements contained in a
>>>> store, I get the following memory error, while it used to work fine
>>>> before:
>>>>
>>>> Array ( [0] =3D> Out of memory (Needed 1042440 bytes) in
>>>> ARC2_StoreSelectQueryHandler )
>>>>
>>>> If I run a more complex query, I do not have this memory problem =20=
>>>> but
>>>> something like:
>>>>
>>>> Array ( [0] =3D> You have an error in your SQL syntax; check the =20=
>>>> manual
>>>> that corresponds to your MySQL server version for the right =20
>>>> syntax to
>>>> use near 'ALL T_0_0_0.s AS `post`, T_0_0_0.s_type AS `post type`,
>>>> T_0_0_3.o AS `d' at line 2 via ARC2_StoreSelectQueryHandler )
>>>>
>>>> I run those queries using the simple $store->query($q); call, and =20=
>>>> the
>>>> config array of the store only contains the DB parameters.
>>>> Is there something else I must to with the new version ?
>>>> I'm running the following version of mysql (on dreamhost shared =20
>>>> hosting):
>>>> mysql Ver 14.12 Distrib 5.0.16, for pc-linux-gnu (i386) using =20
>>>> readline 5.0
>>>>
>>>> Thanks a lot for any help.
>>>>
>>>> Alex.
>>>>
>>>>
>>>
>>>
>>
>>
>
""" ;
ns1:returnPath "<apassant@gmail.com>" ;
ns1:xOriginalTo "arc-dev@semsol.org" ;
ns1:deliveredTo "web11p1@p15192371.pureserver.info" ;
ns1:received """from ?192.168.0.4? ( [82.239.211.193])
by mx.google.com with ESMTPS id e8sm5433135muf.0.2008.05.11.09.43.42
(version=TLSv1/SSLv3 cipher=RC4-MD5);
Sun, 11 May 2008 09:43:43 -0700 (PDT)""" ;
ns1:dKIMSignature """v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;
h=domainkey-signature:received:received:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer:sender;
bh=qXIbracMM0wSETbOrlgEOHKYk+BDkCfAAIv0u9+dcfY=;
b=VqE0kRCczelNB+K4U/ixE0ZJ6J5UB+cmUZmn+njjLoEHGjnGQjFvM1zijFaL9SEVkyML+/hesHRf4wAVRyvlQqVm5AFTg8fPCb5sunz3BvATUtT7qTvVDvloTI9650lu7q04h9X7c2eRJKGAq+N7JVvre8830+JsMh9phmhAgq8=""" ;
ns1:domainKeySignature """a=rsa-sha1; c=nofws;
d=gmail.com; s=gamma;
h=message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer:sender;
b=iVRGNHLN2ZCTfwtB/ap/BiziTxg8T8Xcyw25W6WxQCnNYvWuPPr4SYZKxFvjkORr8Z1A42XcS3vIaO/1+MbwFo+7DkyPIhsib4oceSVOeRgM14VbnS3pneFeIAyC1Gyg+6kU8I5jQ0pmZ+pAZf7XQjWkhQvEH0ei7bhiNoPw7hQ=""" ;
ns1:messageId "<C90B190B-2E1D-46B0-967B-2B6CAB91B7B1@passant.org>" ;
ns1:from "Alexandre Passant <alex@passant.org>" ;
ns1:to '"arc-dev" <arc-dev@semsol.org>' ;
ns1:inReplyTo "<PM-GA.20080508135819.8F6C6.1.1D@semsol.com>" ;
ns1:contentType "text/plain; charset=ISO-8859-1; format=flowed; delsp=yes" ;
ns1:contentTransferEncoding "quoted-printable" ;
ns1:mimeVersion "1.0 (Apple Message framework v919.2)" ;
ns1:subject "Re: [arc-dev] MySQL errors" ;
ns1:date "Sun, 11 May 2008 18:42:50 +0200" ;
ns1:references "<f0e160660805070252k597f92f0ha2b13817d010b28f@mail.gmail.com> <PM-GA.20080507125246.1CA05.1.1D@semsol.com> <f0e160660805071247y25c05fbbt2da3b4a3660dc855@mail.gmail.com> <PM-GA.20080508135819.8F6C6.1.1D@semsol.com>" ;
ns1:xMailer "Apple Mail (2.919.2)" ;
ns1:sender "Alexandre Passant <apassant@gmail.com>" ;
ns1:xSpamCheckerVersion """SpamAssassin 2.64 (2004-01-11) on
p15192371.pureserver.info""" ;
ns1:xSpamLevel "" ;
ns1:xSpamStatus """No, hits=-4.8 required=5.0 tests=BAYES_00,RCVD_IN_SORBS_DUL
autolearn=ham version=2.64""" ;
ns1:xUIDL """4!'#!"":"!el1"!D?""!
