Twenty-five leading mathematicians signed an open letter arguing that AI labs are threatening their intellectual work as they seek to one-up each other with solutions to famous math problems. Each signatory has been awarded the Fields Medal, considered the most prestigious prize in mathematics.
…
While the ability of AI models to solve the world’s outstanding mathematical challenges could be a boon to humanity, the signatories of the new letter argue that will only be the case if those solutions can be understood and communicated by the math community and, ultimately, the rest of the world.
“Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others,” they wrote — and OpenAI’s proof remains unverified. “As in all creative professions, this raises severe attribution and plagiarism questions. Moreover, without the willing mathematicians who must take care of their development and integration into the mathematical canon, AI-conceived ideas would never become fully alive and the crucial human transmission chain between mathematicians would be lost.”
With other mathematicians growing paranoid and wondering if their Codex use was in turn fed into OpenAI’s new models, there is real fear that the culture of open research will be threatened. Today, if frontier labs see a useful path to a discovery, they can spend tens of millions of dollars using LLMs to beat the original researchers to a proof — a dynamic that will incentivize secrecy.
This letter follows the Leiden Declaration, released by a working group of mathematicians in June. That document also grapples with the ways that LLM proofs will change their work and offers a set of recommendations for mathematicians, institutions, and policymakers.
As with software engineering and other areas where AI tools are changing workflows, mathematicians find a justification in the work around the work: The value in math isn’t just the proofs and who gets credit, but the intellectual super-structure that nourishes students, finds new questions and ideas, and integrates them into broader human civilization.
And if you don’t particularly care about the cutthroat world of high-stakes mathematical proofs, don’t forget: Your field of interest is next.
“The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place,” they wrote.
We need to stop giving AI credit for anything as if it’s a person. AI is a tool. When someone uses AI to perform a task, then the user gets credit, much in the way a seamstress gets credit for
sowingsewing a dress using a Singer. Neither the sewing machine nor Singer get credit for the gown.Now this doesn’t solve everything, since AI operates by taking scrapes of the internet and media, and uses this to derive a solution, the process should also include citations for all its sources, just as any proper research project would.
And so if an AI uses someone else’s proof (or partial proof) to inform its own, it should be required to provide citation. Otherwise, it’s up to other academics to repeat the research to verify what the AI did, and then they get credit, as do all cited sources.
Pretty sure the whole point of this is to “we need to start giving ai credit” from the other side.
"arguing that AI labs are threatening their intellectual work as they seek to one-up each other with solutions to famous math problems. "
That a disingenuous way of describing research.
Also, the math guys have a point. A solution or proof that nobody understands has no value.
AI is doing to math what it does everywhere else, smear it with excrement then declare success.
And as anyone that has used AI will tell you, it is incorrect 50 percent of the time. If you can’t check it, how can you trust it.
My understanding is that they published a machine-checkable proof (Lean).
There’s already valid epistemological question about whether proofs that are only machine-checkable are the same quality of knowledge as traditional proofs. For instance, the 1976 proof of the four-color theorem was controversial in part because it was too long to be read by a human.
Another part of the argument is that the foundations are quicksand: proof checkers include unverified code (read: potentially buggy) and run atop unverified OS kernels on a likely proprietary CPU! (Personally, I think if you can understand the theorem statement, it’s safe to delegate the drudgery of the proof-checking to a computer. If two different systems agree, that’s good enough for me. But I’m neither a mathematician nor a philosopher.)
These AI generated proofs push the boundary further because humans participated even less in the formalization and proof development.
Im not an AI lover, but this critique that it is wrong 50% of the time is simply not true for newer models.
It’s still consistent with my experience, but in certain contexts, can be corrected if the output can be processed in some context that can auto detect wrong output. For example if you have a unit test to pass, then it can be wrong several times but doesn’t matter because it auto runs and gets directed to try again.
In contexts without the possibility to objectively measure correctness, it is still pretty bad.
Incidentally that’s why I don’t let it go crazy on anything without tests. Shit, it could rerun the test 10 times if it needs to, I don’t really care what the intermediate steps are, just that the end result is correct and it isn’t polluting my codebase with 7 line comments and docstrings for helper methods that are simpler than left-pad. If it then successfully fixes a difficult-to-find (but often easy to fix) bug that stems from changes to a large upstream codebase 9 times out of 10, then as far as I’m concerned, the accuracy of the tool I’m using is 90%. Don’t much care if the underlying model’s accuracy is technically only 50% or maybe even less. Still haven’t found a definitive solution for the claudespeak comments. They’re annoying and no matter how and how many times I tell it to stop adding them, it will fucking not.
Which is ALSO why I think asking a chatbot for advice on anything you’re not knowledgeable about yourself is a horrible idea. The accuracy is OK if it can iterate against validation.
I still remember trying to get it to solve a problem I had with nested multi-level navigation bars in an enterprise app, and it’s JavaScript solution was like:
const nav = document.querySelector('nav'); window.multiLevelNavBar(nav);Hey LLM, I got an error;
multiLevelNavBardoes not exist. What the fuck?Oh my gosh, you’re right, that’s 100% on me. Here’s the REAL solution… (Insert some more fake bullshit)
It’s good at filling in the blanks. But if the blanks are too big, all you get is filler
Id say its better than majority of my colleagues. Can you imagine if we fact check people in the same way?
Depends. It is way often more right than it is wrong, however if you have a long enough chat, the chance that it produces a falsehood approaches 100%. Of course humans produce errors too.
Too some extent I agree with you. However, most of the time you really dont need to have “long” chats. In some ways I think some of the Flasehoods derive from the prompts themselves. Lots of bad/false/incorrect info is available on the web.
Can you imagine if we fact checked long conversations with people? How often do thing bad/false info would surface?
Do you not fact check your conversations?
If one person in the group has any sniff of not true, someone pulls out their phone to check.
Friends dont let friends spread misinformation. Fact check today.
Most of the conversations I’m talking about are not a quick whip out the phone and check…
Yeah, if you pay, you’re right. But how often would you say the newer models are incorrect? Even if it’s 1%, that still hold true that you can’t check them.
Id say its accurate more than 90% of the time. I see erroneous facts added to the response that are wrong or not correctly represented the most.
Do you apply this same logic to the “experts” that you talk to? I talk with alot of SMEs from different domains. I could never fact check them all. Way too broad of subject matters. Similar to many of my indepth AI interactions
Do you apply this same logic to the “experts” that you talk to?
Yes, I do. Especially that they can have their shit peer reviewed outside of academia.
Usually when I do this after the fact it comes back as “oh, that specific context wasn’t given” or “oh, I misunderstood what you were asking”, etc. Seems like the same…
Humans can’t check their work though? That’s the issue. Science is the process of checking the work. AI just assumes that you’re getting the correct version with no future checking. Think Fermilab or Cern.
I’m not so sure that human understanding a single proof is critical. Sometimes a proof serves as a stepping stone to other proofs. Understanding is important, but becoming another tool is also important. So imagine if there was a proof, even human-generated but not fully understood, to Riemann zeta’s zeros. That, even if not understood fully, would lead to other important maths.
What is the point of a proof if it can’t be understood?
To facilitate other proofs that both can be understood and lead to other advances.
Not trying to be difficult here, I don’t have a mathematical background.
How can an unverified, non-understood proof be used as foundational for any other work?
It can’t. But I was replying to
A solution or proof that nobody understands has no value.
That implies a different meaning, at least in that person’s context, between proof and human understanding, with which I agree.
An unverified proof is not a proof; it is just conjecture. But there can be a rigorous, fully verified proof that we don’t fully have that intuition to understand. Those can be created by humans even, leaving AI completely out of the picture.
All I’m saying is a rigorous proof, even if not fully understood can be useful to finding and understanding other maths.
So you just build a black box that pops out numbers?
You need to understand all the proofs in the chain so you can verify they work. It’s a thing you learn in grammar school when they teach you to write the steps down to a solution instead of just the final result.
Oh, if your metric is that understanding and rigorous verification are the same then I agree, but then there is no proof at all without human understanding, so what are we talking about?
In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas.
I think this is a really important point. Even if A.I. “could” theoretically (big theoretically) solve all current problems, I don’t think it could solve future problems. Without humans, students, etc, asking new questions with curiosity and creativity; it would just be a hard cut off point where innovation ends.
LLMs were trained on stealing past data, what happens if they push out humans completely from an industry? There would be no more data to train on, and all possible skills in that area would die.
… the whole point of solving those problems is to discover what we were missing in the process.
You can’t just “solve” one of these problems and say “see, all good.”
Here’s the letter: https://mathandai.org/
I think that they know that showing the chats would reveal they spied on the mathematician’s methods and that they basically stole the idea and front runned the discovery.
At least some people realize that AI is just straight shit
