+- SphereCommunity (https://forum.spherecommunity.net)
+-- Forum: Sphere 0.56d (/Forum-Sphere-0-56d)
+--- Forum: General Help (/Forum-General-Help)
+--- Thread: Huge tags = sphere crash (/Thread-Huge-tags-sphere-crash)
Huge tags = sphere crash - Coruja - 04-28-2014 03:21 AM
If this huge tag got stored on a char, the entire server will crash when sphere try to read it again.
Quote:TAG.Sync.Paperdoll="layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer22=NULL, layercor22=NULL, layer22=9860, layercor22=1940, layer2=5122, layercor2=NULL, layer2=NULL, layercor2=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer2=5122, layercor2=NULL, layer2=NULL, layercor2=NULL, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer20=NULL, layercor20=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer2=3912, layercor2=NULL, layer2=NULL, layercor2=NULL, layer2=3912, layercor2=NULL, layer2=NULL, layercor2=NULL, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713"(this tag is just to store all equip/unequip on the char, to later execute a single SQL command instead execute another command for each one)
On windows I got the error "sphereSvr.exe stopped and must be closed" and no console error on sphere. I took the entire week to find this problem since sphere force-close without a single console error.
Thats a huge problem, because I can fix the script myself, but it can occour again at any time on any other script, or occour on any other server. And if occour, ppl will got their servers crashed at every moment without know why
So even fixing on script, sphere must have a internal protection for this thing too, it will save many servers to close (nobody wants to play on a server that crash 10x each day)
I have no idea how this internal fix could be done, but something like this will help:
1) Add support for huge strings (sphere will read it correctly instead force-close sphereSvr.exe)
2) Add a length limit on tags (my tag has 1768 chars and crash the server, so it must be something less than this, maybe 1024 or 1500). If exceeded, show a console error "ERROR:(script.scp,666): Trying to write a TAG with length > 1500"
RE: Huge tags = sphere crash - Alaric - 04-28-2014 03:45 AM
Uh, I rewrote tag which had lenght of +-100 chars because i thought i was too long. I used list or flag method/bitmask operations instead.
You broke my record man
This tag is damn ugly.
RE: Huge tags = sphere crash - Feeh - 04-28-2014 04:14 AM
I was not able to reproduce this, no crashes
The script line length is limited to 4096, so no problems here (Sphere built on 23/apr/2014)
Is the problem occurring when the script ask to update the SQL table? Try to isolate the tag/data collection in one function then the SQL update into another function if not already done
Code:
[function test]
tag.test
for x 1 2000
tag0.test .= "h"
serv.log <dlocal.x> - <EVAL STRLEN(<tag0.test>)>// - <tag0.test>
endfor
TAG.Sync.Paperdoll="layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer22=NULL, layercor22=NULL, layer22=9860, layercor22=1940, layer2=5122, layercor2=NULL, layer2=NULL, layercor2=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer2=5122, layercor2=NULL, layer2=NULL, layercor2=NULL, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer20=NULL, layercor20=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer2=3912, layercor2=NULL, layer2=NULL, layercor2=NULL, layer2=3912, layercor2=NULL, layer2=NULL, layercor2=NULL, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713"RE: Huge tags = sphere crash - Coruja - 04-28-2014 05:14 AM
this tag is used on my SQL paperdoll sync engine
all @Equip/@Unequip got stored on this tag to later call a TIMERF (2~4 seconds) and run a single SQL command to everything stored. This engine is used to prevent overload on SQL server using another new command for each @Equip/@Unequip
function to reproduce the exact behavior:
Quote:[FUNCTION lol]
TAG.Sync.Paperdoll="layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer22=NULL, layercor22=NULL, layer22=9860, layercor22=1940, layer2=5122, layercor2=NULL, layer2=NULL, layercor2=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer2=5122, layercor2=NULL, layer2=NULL, layercor2=NULL, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer22=NULL, layercor22=NULL, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=2207, layer22=9860, layercor22=1940, layer8=NULL, layercor8=NULL, layer8=4234, layercor8=71, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer20=NULL, layercor20=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer2=3912, layercor2=NULL, layer2=NULL, layercor2=NULL, layer2=3912, layercor2=NULL, layer2=NULL, layercor2=NULL, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713"
LOCAL.SQL=<DB.AEXECUTE SQLAsyncResponse, UPDATE paperdoll SET <TAG.SQLSync.PaperdollItem> WHERE uid='<UID>' LIMIT 1>
[FUNCTION SQLAsyncResponse]
IF !(<ARGN2>)
SERV.LOG @SQL error: <ARGS>
ENDIF
RE: Huge tags = sphere crash - Ben - 04-28-2014 05:36 AM
I'm curious how your database is laid out... I don't think it's normal to have multiple cols with the same name.
More then likely the issue would be with SQL.
RE: Huge tags = sphere crash - XuN - 04-28-2014 05:39 AM
LOCAL.SQL=<DB.AEXECUTE SQLAsyncResponse, UPDATE paperdoll SET <TAG.SQLSync.PaperdollItem> WHERE uid='<UID>' LIMIT 1>
Do you notice the comma in SQLAsyncResponse, ?
RE: Huge tags = sphere crash - Coruja - 04-28-2014 06:06 AM
(04-28-2014 05:36 AM)Ben Wrote: I'm curious how your database is laid out... I don't think it's normal to have multiple cols with the same name.the SQL table has 48 fields (2x 1~24)
More then likely the issue would be with SQL.
layer1, ..., layer24 (store item ID on each layer)
+
layercolor1, ..., layercolor24 (store item color on each layer)
if the SQL command use "layer1=666, layer1=NULL, layer1=2000", it's not a triplicated layer1 field, but a optimization to save performance because instead execute the same command 3x to update the same field value, it will execute only a single command with all values and use only the last (2000).
(04-28-2014 05:39 AM)XuN Wrote: LOCAL.SQL=<DB.AEXECUTE SQLAsyncResponse, UPDATE paperdoll SET <TAG.SQLSync.PaperdollItem> WHERE uid='<UID>' LIMIT 1>yea, I'm using this same SQLAsyncResponse on many others SQL command (sync kills, deaths, karma, fame, many account logs, etc) and all of them works fine. Including this same function (paperdoll sync) is working fine. The only problem is this huge tag
Do you notice the comma in SQLAsyncResponse, ?
About ~6 months ago I already got this identical problem with TAGs crashing the server using a "virtual keyboard system" which doesn't use SQL. All virtual keyboard inputs was stored on a TAG, and if the player press 813508365081015 keys, the tag will get huge and crash the server. So I'm quite sure this is not a SQL related problem
RE: Huge tags = sphere crash - XuN - 04-28-2014 06:49 AM
The comma is sticked to the function, hence giving error:
Code:
[function test]
local.sql=<SQLAsyncResponse, UPDATE paperdoll SET <TAG.SQLSync.PaperdollItem> WHERE uid='<UID>' LIMIT 1>
[FUNCTION SQLAsyncResponse]
IF !(<ARGN2>)
SERV.LOG @SQL error: <ARGS>
ENDIFResult: 22:47:ERROR:(sphere_defs.scp,15)Can't resolve <SQLAsyncResponse, UPDATE paperdoll SET WHERE uid='01' LIMIT 1>
And for the tags...I didn't test yet.
RE: Huge tags = sphere crash - Coruja - 04-28-2014 07:07 AM
try using DB.AEXECUTE before SQLAsyncResponse
Quote:[FUNCTION test]
LOCAL.SQL=<DB.AEXECUTE SQLAsyncResponse, UPDATE paperdoll SET <TAG.SQLSync.PaperdollItem> WHERE uid='<UID>' LIMIT 1>
[FUNCTION SQLAsyncResponse]
IF !(<ARGN2>)
SERV.LOG @SQL error: <ARGS>
ENDIF
RE: Huge tags = sphere crash - Ben - 04-28-2014 10:18 AM
Are you sure it wouldn't be a problem with tags not matching?
TAG.Sync.Paperdoll="layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer1=5184, layercor1=NULL, layer2=7030, layercor2=2057, layer6=NULL, layercor6=NULL, layer6=5138, layercor6=2057, layer1=NULL, layercor1=NULL, layer2=NULL, layercor2=NULL, layer6=NULL, layercor6=NULL, layer6=5447, layercor6=713, layer22=NULL, layercor22=NULL, layer22=9860, layercor22=1940, layer2=5122, layercor2...
LOCAL.SQL=<DB.AEXECUTE SQLAsyncResponse, UPDATE paperdoll SET <TAG.SQLSync.PaperdollItem> WHERE uid='<UID>' LIMIT 1>
Using your code and a simple debug message, it showed me that it was trying to "UPDATE paperdoll SET 0 WHERE uid='...' LIMIT 1"
I did not actually do the AEXECUTE here, I was simply trying to see if the query was properly formed.
Now I'm not an SQL expert here, but I don't think doing SET 0 is proper syntax... and doing so inside an async query is probably not very good... So I'm gonna take a wild guess here and say the issue might be with async database handling not being able to handle USER mistakes properly.
When it comes to Sphere string handling, I think we are ok.
Any strings can hold up to 2147483647 characters (the 4096 Feeh mentioned is only for SCRIPT lines but it's still higher then what you need)