AI moderation: a mistaken block needs a path to review
A model signal is not yet a decision to hide a post. Examine the moderation rule, context, mistaken blocks and actual restoration.
Article language: English

In this article
A comment disappears. Its author is told it broke the rules, while a moderator sees a category returned by a model. Nobody recorded which site rule connected that category to hiding the comment. When the author asks for review, the system runs the same model again and shows the same answer. The appeal has completed a very efficient circle.
This is a hypothetical film-review site. Some comments describe plots, some debate with other people and some call for intervention under community rules. Evaluating this use of AI should cover more than assigning a category. Follow what happened to the particular post after the result and how a mistake can be corrected.
An owner's rule connects a category to an action
A model can indicate the presence of a type of content. The site owner decides whether that result means allowing it, referring it for review, hiding it temporarily or taking another action. Record that connection and the person responsible for the rule separately. The team can then distinguish a wrong classification from a valid signal applied under the wrong rule.
The Azure AI Content Safety FAQ explains that the service returns classification metadata and does not itself remove content or ban platform users. Subsequent action follows the policies of the application using the result. This is a boundary of responsibility for that service, useful when describing the whole moderation process.
On our site, a mention of violence in a film review need not be an attack on another participant. Community rules should determine what material is allowed, in which context and with what presentation. A general requirement to "be safe" cannot replace that decision. A moderator needs rules under which they can justify retaining or changing visibility.
A provider's categories need not cover every community rule. A film site might require a warning on a comment that reveals the plot. A low score in a harmful-content category will not confirm compliance with that separate rule. At acceptance, record which rule is handled by the model, which by another mechanism and which by a moderator. A missing flag will then not be given the meaning "everything checked" when the actual scope was narrower.
OpenAI's documentation recommends treating moderation scores as signals for an application's policy rather than an automatic blocking decision. It distinguishes category flags from numerical scores and notes that custom rules using scores may need recalibration after model changes. It is not a ready-made rulebook for our community.
For acceptance, retain the category and the version of the rule that led to the action. A number needs a name and a defined meaning. As with interpreting AI confidence as a percentage (in Polish), the appearance of a score alone does not prove its accuracy for the company's task. A score for one category is not a universal percentage of violations of every site rule.
Also record whether moderation happens before publication, after publication or after a report by another person. These points have different consequences. Content awaiting review should not look published to its author if only they can see it. After a previously visible comment is hidden, it is necessary to confirm that the change concerns the right version of the post.
An author may edit a comment while its previous version awaits review. A result concerning the old content should not silently determine the new version. Link the assessment to an identifiable version and the time it was submitted. When an appeal is opened, the moderator should know whether they see the originally hidden post or a later correction. Otherwise they may justify restoring text the system never actually assessed. How necessary material is retained and who may access it must be agreed in the company's process.
During review, an author may also delete the comment. A favourable decision should not then automatically republish it against its current state. The moderator needs to know about the deletion and have an agreed rule for closing the case. Changing the assessment and showing the content again are separate actions. Include this case alongside ordinary restoration in acceptance so that correction does not recreate a post the author has withdrawn.
Test cases must retain what the post means
Prepare cases with expected decisions under the site's rules. In our fictional community these include a description of a film scene, a critical opinion about a work and a statement addressed to another person. A report available to a wide team need not publish the full wording of harmful material. Its role in a controlled test can be identified while access to the material is appropriately restricted.
Context should be part of the same input the real process will use. A sentence taken out of a reply may contain a quotation the author condemns. In another thread a similar string of words may serve a different purpose. The evaluation team should know whether the model received only the comment, the previous message, the film title or other information.
Do not add hints to tests that are absent from ordinary work. A phrase such as "this is an acceptable quotation" beside the text may make assessment easier, but changes the conditions of the task. For every comparative pair, retain a description of the available context. If the process is meant to recognise quotation, the material must make that distinction testable.
Your reference answer needs justification. When two people interpret a rule differently, record the disagreement and resolve its source. A definition, example or boundary between criticism and behaviour not allowed in that community may be missing. Assigning one arbitrary label would create a falsely certain reference against which the model is then compared.
Evaluating classification of business messages (in Polish) helps organise category errors. Moderation also requires evaluating the effect on a post's visibility. A valid signal can lead to an incorrect block when the action rule does not account for allowed context. The report should show that stage so the correction addresses the right cause.

Separate mistaken hiding of permitted content from leaving up content that should have been stopped. One average accuracy figure can conceal the direction of errors. Also check whether errors cluster in a particular language, type of post or thread length used by the company. A sample of Polish reviews does not automatically establish moderation quality in every language on the site.
Acceptance must cover technical states too. If the classification service does not respond, do not call that a detected violation. The process owner should choose how to handle the interruption, and the author should see the true state. Deferring publication to a queue can be an agreed decision, but it needs its own message and subsequent handling.
Review must be able to change the outcome
The author should recognise which post is affected and how to request review. "Content violates the rules" without naming the relevant rule does not help them add context. An explanation can be short and suited to what the site can disclose. It should not attribute a final decision to the model if the application rule made it.
Microsoft's Azure AI Content Safety transparency note recommends meaningful human review to resolve false flags and help people contesting a result. It also calls for feedback channels. These are integration recommendations for the service; your process should name the responsible person and the condition for closing a case.
Do not call a repeat of the identical request a review. It may provide another observation, but does not create an independent decision under community rules. The moderator should receive the post, necessary context, the signal used and the action rule. They must be able to retain or change the decision with a reason.
An author's request can challenge two different stages. "This is a quotation describing a film" questions the interpretation. "The rules allow such a description with a warning" concerns the rule applied. The review form should retain that distinction and allow necessary context without requiring the whole comment to be rewritten. The relevant material should already belong to the case. Giving a moderator only a numerical result and "the user disagrees" removes information needed for a decision.
A generated account of the reason is not evidence of how a decision was made. The difference between an AI explanation and an actual trace of its action (in Polish) has a practical effect here. If a report adds an elegant reason only after hiding, do not treat it as a reconstruction of the earlier rule. A record of the actual process is needed.
The appeal queue needs an owner. "Sent to a moderator" should correspond to a real case in a managed exception queue. The guide to exception queues (in Polish) discusses how such a queue can be organised. Without an assignment and a path to closure, requests can be collected efficiently and never resolved. A count of new appeals does not yet say how many mistakes were fixed.
Check the entire correction: the decision changed, the post regained the appropriate visibility and the author received information matching that state. Closing a task in the moderator's panel alone does not prove the comment was restored. If the public view still hides it, the error remains from the author's perspective.
Restoration should also survive the next automatic processing of the same version. Without an agreed rule, the comment may return only briefly before a later run hides it again. This does not mean exempting the author from moderation forever. The decision concerns a particular post and agreed scope. Editing it or changing the rules may require a new assessment. Test both cases so a correction creates neither a blocking loop nor an invisible exception for all future content.
In an Aeneia Audit review of the moderation process, rules, samples and consequences can be connected to a route for correcting mistakes. Community rules, example cases and a record of actions taken form the material for discussion. If no restoration path exists, the first decision is who performs an approved correction and where.
Revisit resolved cases when a rule changes
A change to a threshold, model or the context supplied requires another pass through the samples. Retain earlier results and identify what changed. If a correction restores permitted quotations, also check cases that were previously stopped correctly. Do not assess the new rule only on the example for which it was prepared.
Before applying a new rule to live comments, consider a shadow-mode comparison (in Polish). The second run records a proposed decision but does not hide or restore a post or send another notification. This action boundary must be demonstrated before such a test starts. The comparison does not replace deliberate false-block cases or separate acceptance of actual restoration.
In the report, distinguish improved recognition from improved handling. A team may still make a similar number of mistakes but resolve them faster and restore content correctly. It may also have a more accurate model while still failing to perform approved corrections. These are two different areas of work and need different evidence.
Appeals alone will not reveal every mistaken block. Some people may not notice the hiding, find the button or continue the conversation. In a controlled sample, also review stopped posts that nobody challenged. Separately record the share of cases completed by review and its outcome. Fewer requests after hiding the button would not prove better moderation; it would reflect a change in how participant feedback is collected.
Do not treat every stop as good caution. As with evaluating an appropriate AI refusal (in Polish), the decision needs a basis and a further path. The author of a critical opinion allowed by the site's rules should not have to wait for a chance gesture from a system that reads the same text once again.
Accept one complete false-block case, from first publication to visible restoration. If the company can show who changed the decision, under which rule and what the author actually saw, it has a basis for evaluating the process.








