Item8942: UID's get clobbered by bad url encoding

We have a large number of users at my university with underscores in their login names. We finally upgraded our production system from T* 4.2.x to Foswiki 1.09, and noticed that author-based query searches started failing. It seems that the uid's written to the author field of META:TOPICINFO post-upgrade are getting clobbered by a bad url encoding attempt.

For example the username smith_j is written to the topic metadata as smith_5fj, which then breaks a variety of fun things.

As Babar pointed out in IRC:

[11:52am] tsnfoo: grepping doesn't show any instances of uri_escape, so somebody is probably doing it manually somewhere.
[11:55am] Babar: core/lib/Foswiki/Users.pm line 1057
[11:55am] Babar:     $cUID =~ s/([^a-zA-Z0-9])/'_'.sprintf('%02x', ord($1))/ge;
[11:56am] Babar: (on trunk, on release branch, you can find it I guess :-D

From the above you can see that certain characters are replaced with an underscore and their ISO-8859-1 hex code.

Questions:

• Why is this even necessary? I assume that someone thought it was worth it to sanitize UID strings as protection against some attack vector, something down the processing chain expects UIDs to be encoded, or they're just safer to pass around that way.
• Regardless of how they're processed mid-flight, shouldn't key metadata elements like UID should always be stored their proper, canonical format?

This is causing a pile up of fail for us, because every topic edit/create for these users results in corrupt topic metadata.

Can anybody take a quick look at where the right place to fix this might be?

-- AaronFuleki - 21 Apr 2010

Its necessary because that string is used for the rcs user - which must be 7bit ascii. CUID (the encoded form) is the canonical internal form of a user's id. It wasn't supposed to be used in query searches (though something tells me that we've not written the user comparison code ). imo for 2.0 we need to add a userequals operator that can do comparisons using wikiname, loginname and cuid interchangabily, but that's a future feature request that needs to be made.

for now I would suggest that you add '_' to the line that Babar shows above to stop it from happening - or if you're 100% positive that your user system won't provide non rcs-able characters, comment it out?

-- SvenDowideit - 30 Apr 2010

As Sven suggests, this needs to be handled through a feature request. Regrading this report from Urgent to reflect that, and confirming it.

-- CrawfordCurrie - 30 May 2010

Commenting is disabled while not logged in