Allow emoji in contribution names - #440
Conversation
A four byte character in a contribution name reaches the utf8 column as-is and MySQL rejects the row, both when the contribution is created and when it is edited. Replace those characters with their numeric character reference in submit(), using the same utf8_encode_ucr() core applies to report text. Encoding in the entity rather than at the call sites keeps the stored form out of everything that reads the name, including the release topic subject, the feed and the generated permalink, and no consumer needs to decode it. Character references are dropped when generating a permalink from the name, so their digits do not end up in the URL.
|
This still does not work because the un-encoded emoji is still trying to be stored to the SQL ERROR [ mysqli ] Incorrect string value: '\xF0\x9F\x86\x95' for column 'contrib_name_clean' at row 1 [1366] SQL INSERT INTO cdb_contribs (contrib_id, contrib_type, contrib_name, contrib_name_clean, contrib_desc, contrib_desc_bitfield, contrib_desc_uid, contrib_desc_options, contrib_categories, contrib_demo, contrib_status, contrib_user_id, contrib_downloads, contrib_views, contrib_visible, contrib_rating, contrib_rating_count, contrib_creation_time, contrib_last_update, contrib_release_topic_id, contrib_faq_count, contrib_iso_code, contrib_local_name, contrib_clr_colors, contrib_limited_support, contrib_package_name) VALUES (0, 8, 'Emoji 🆕', 'emoji_🆕', 'Emoji <EMOJI seq="1f195" tseq="1f195">🆕', '', '', 7, '18', '', 1, 2, 0, 0, 1, 0, 0, 1785159808, 1785159808, '', '', '', '', '', 0, '') However, I'm starting to think we should not allow emoji in contrib names. Emoji should only be allowed in descriptions. Contrib names should remain alpha-numeric IMO. |
The permalink is generated during validation, before the name is encoded, so a name containing an emoji still reached contrib_name_clean as-is and MySQL rejected the row: Incorrect string value: '\xF0\x9F\x86\x95_s...' for column 'contrib_name_clean' at row 1 [1366] Drop character references and characters outside the basic multilingual plane in generate_slug() rather than in generate_permalink(). A slug has to be safe for a URL and storable in a utf8 column whichever way round it is called, and every caller benefits, including the topic subject slug. The strip added to generate_permalink() is no longer needed and is removed.
|
Good catch. The permalink is generated in Now dropping four byte characters and character references in |
|
Check out #441 as a more complete solution, possibly |
Picks up #390, which has been sitting as a draft since 2022. The diagnosis there is @battye's, and the approach here is @iMattPro's suggestion on that thread.
A four byte character in a contribution name reaches the
utf8column as-is and MySQL rejects the row, on create and on edit both:utf8_encode_ucr()replaces those characters with their numeric character reference, the same thing core does to report text inphpbb\report\controller\report. Encoding intitania_contribution::submit()rather than in the controllers covers create and edit together, and keeps the stored form out of everything that reads the name, so nothing has to decode it.That last part is why this does not take the approach in #390. Parsing the name leaves s9e markup in the field, which then reaches the queue topic subject, and @battye ran into exactly that in June 2024:
A character reference has neither problem. Dropping them when generating a permalink keeps the digits out of the URL, otherwise the slug comes out as
cool_127381;_extension.Checked on a local board:
cool_extension#439 fixes the attention fields, which fail the same way, using the same function.
If @battye would rather carry this on #390 then I am happy for this to be closed in favour of it.