How a claim earns the right to be called a result.

Hypothesis formation

Research starts from an observation, but an observation only shows that something happened once. Before any trial, the study records the question, the behavior that would count as a positive or negative answer, and the hypothesis if there is one. Competing explanations are written down at the same time, because they determine which controls the experiment needs.

Analyses chosen after the data is seen are allowed, and are labeled as exploratory. They can motivate a new study. They cannot confirm the hypothesis that the data was collected to test.

Experimental design and controls

• Conditions are varied one at a time where the question allows it, so that an effect can be attributed to a cause.
• Positive controls show that the test can detect the failure it looks for. A test that never fires proves nothing.
• Negative controls and baselines show how often the behavior occurs without the manipulation.
• Equivalent conditions are required for any comparison between systems: the same inputs, the same instructions, the same scoring and a recorded date for each run.

Failure reproduction

A failure observed in the wild becomes a research case only when it can be reproduced. The reproduction records the minimal conditions under which the failure occurs, the conditions under which it does not, and the rate at which it recurs across repeated attempts. A failure that cannot be reproduced is reported as an observation, with that limitation stated.

Test design and scoring

Scoring rules are fixed before results are scored. Where judgment is needed, the study publishes a rubric with definitions and worked examples for each level. Where more than one rater applies a rubric, agreement between raters is measured and reported, and disagreements are resolved by a documented procedure rather than by discussion of the expected answer.

Evidence preservation

Outputs are kept verbatim with the inputs and settings that produced them. Evidence is never paraphrased in place of the original. When material must be withheld or sanitized for privacy, confidentiality or safety, the publication says what was removed and why, and limits its claims to what a reader can still check. Working exploit payloads are never published.

Statistical treatment

Every rate is reported with its numerator and denominator. Small samples are described as small, and results from them are not generalized beyond the conditions tested. Where a study makes a quantitative comparison, it reports the uncertainty around the estimate and the method used to compute it. Repeated trials are planned in advance so that their number is not chosen after seeing which way the results fall.

Versioning and repeatability

AI systems change without notice. Each trial records the system name, the version string where one is exposed, the access date, the interface used and the relevant settings. A result is stated as a result about those versions. Publications have stable identifiers and dated revisions, so a later reader can tell which version of the text made which claim.

Independent replication

A study is called reproducible only when its materials and instructions allow another investigator to perform the checks, not because its prose is detailed. ALGEDI invites replication attempts, publishes them, and labels a result as independently replicated only when a separate investigator has reproduced it and the evidence is linked.

Disclosure of uncertainty and limitations

Every report separates four things: what was directly observed, how it is interpreted, what remains hypothesis, and what is unresolved. Limitations and known confounds are part of the publication, not an appendix to it. Claims about a model's internal reasoning, intentions or consciousness are outside what behavioral evidence can establish, and are not made.

Publication states

Each entry carries exactly one state. The state is validated when the site is built: an entry cannot claim results it does not contain, and a replication cannot be claimed without its evidence.

Corrections and retractions

When an error is found, the publication is corrected in place and the correction is dated and described in its record. If an error invalidates a conclusion, the entry is marked as corrected or archived, and the original claim remains visible alongside the correction. Nothing is silently removed. Reports of errors are welcome through the collaboration page and are acknowledged in the record when they lead to a correction.

See also the research integrity and disclosure policy.