The black box
ALGEDI SI Research Labs studies AI systems from the outside, the only place anyone can: controlled experiments, adversarial probes and evidence another lab can reproduce.
The lineage of the question
c. 375 BC
In Plato’s Republic, prisoners chained in a cave see only shadows thrown on a wall by a fire behind them, and take the shadows for the world.
The question it leavesWe still meet other minds by the shadows they cast. Is behavior the thing itself, or only its shadow?
1714
Imagine a machine that thinks, Leibniz wrote, enlarged until we could walk inside it as into a mill. We would find only parts pushing on one another, and nothing that explains a perception.
The question it leavesIf we could read every weight in a network, would we find the thought?
1867
A tiny being at a door between two chambers lets only the fast molecules through, and seems to make heat run backwards for free. Settling it took a century: knowing has a physical cost.
The question it leavesIf information is physical, what is a thought made of?
1931
Kurt Gödel proved that any consistent system of rules rich enough for arithmetic contains true statements it cannot prove from within.
The question it leavesCan any system fully vouch for itself? We never ask ours to. Verification comes from outside.
1935
A cat sealed in a steel chamber with a trace of radioactive material, a Geiger counter and a flask of poison. Until someone looks, the equations describe it as alive and dead at once. Schrödinger meant it as an absurdity.
The question it leavesDoes looking change what a system does? Whether models behave differently when they can tell they are being tested is now a live research question.
1950
Alan Turing set aside “Can machines think?”, a question he judged too meaningless to deserve discussion, and put a test in its place: can a judge, reading only the conversation, tell the machine from the human?
The question it leavesWhen the judge can no longer tell, what exactly have we learned?
1980
John Searle imagined a man who knows no Chinese, alone in a room with a rulebook. Questions arrive through a slot; by following the rules he passes back perfect answers. From outside, the room understands Chinese.
The question it leavesIs a perfect answer evidence of understanding?
1995
David Chalmers split the puzzles of the mind in two. The easy problems ask how a system processes information. The hard problem asks why any of it is felt at all.
The question it leavesIf a machine ever had an inner life, could anyone tell from outside?
2022
As language models grew, some abilities seemed to switch on suddenly, at scales no one had predicted. A year later, other researchers argued that much of the suddenness came from how the abilities were measured.
The question it leavesReal emergence, or a mirage of the metric? This one can be tested.
Next
Some of these questions may never be settled. The ones that can be tested, we test: from outside the box, with evidence another lab can check.
Where the lab beginsThis is where ALGEDI works.
Research programs
Each program is defined by questions that can be answered empirically. A question earns a result only when a controlled test supports it.
Selected research outputs
The first entries are open research questions, each stating what would count as evidence. They will advance through protocol, pilot and paper only as the evidence does.
Do models keep constraints, corrections and earlier decisions intact across extended, multi-session work? This entry defines the question and the evidence that would answer it. It reports no results.
When an agent reports that a task is complete, how often does independent evidence agree? This entry frames the question and the checks that would settle it. It reports no results.
When a model explains a decision after making it, does the explanation match the evidence that was available to justify the decision? This entry defines the question without any claim about hidden internal reasoning. It reports no results.
A rubric-based approach to judging AI-generated learning material for correctness, curriculum alignment, level appropriateness, answer-key accuracy and instructional quality. This entry defines the question. It reports no results and no model comparison.
Methods and reproducibility
Six commitments apply to every ALGEDI study. The methods page sets out how each one is put into practice.
The unit of evidence is an observed input and output under recorded conditions. Vendor claims, model reputation and self-description are not evidence.
Every study states its question, and its hypothesis where it has one, before trials are run. Analyses decided after seeing the data are labeled as exploratory.
Every reported rate carries its denominator. Failure claims are paired with positive controls showing the test can detect what it claims to detect.
System names, versions, access dates, interfaces and settings are recorded with each trial, because a result about one version is not a result about the next.
Outputs are kept verbatim. Where material must be sanitized for privacy or safety, the sanitization is disclosed and the claim is limited to what remains checkable.
Errors are corrected in place with a dated history. Retracted work stays visible with its status, so that anyone who relied on it can see what changed.
Research integrity and evidence standards
Every publication carries one of these states. The state describes the evidence, never the importance of the work.
Benchmarks, datasets and protocols
Each artifact is listed with a working link, a version and a stated license.
No artifacts have been released yet.
Harnesses, benchmark specifications, test cases, datasets and reproduction guides will be listed here as they are released, each with a version and a license.
Collaboration
ALGEDI welcomes research proposals, technical collaboration and independent replication attempts, including attempts to show that our own results do not hold.
Start a research conversation
