At DEFCON on August 11, Chief Digital and AI Officer for DoD Dr. Craig Martell spoke about large language models (LLMs) and their future use at the Department of Defense (DoD). In that talk he said the following about the need for accuracy: “I need five nines [99.999% accuracy]
GPT-4 successfully took the test we use as a technical screen for hiring software developers and data scientists. With almost no human judgment necessary, it was able to score 100% on the developer test and nearly that high on the data science one. I’m going to discuss what our
Last year, when I was looking at a publicly-released Excel file on school test scores that had been released by the Office of the State Superintendent of Education (OSSE) for the District of Columbia, I guessed that something was not working as intended. While much of the data was suppressed
When organizations release data but redact or suppress certain parts of it, like information pertaining to small subgroups, some information they reveal can let viewers infer the information that was hidden. One of the things that can give away what they were trying to keep secret is percentages. If you
