OpenAI's 175B-parameter autoregressive language model introduced via research paper in 2020, notable for strong few-shot performance across NLP tasks without fine-tuning.
@hn_b5dbbe
27 days ago
Overpromising? Underpromising, if anything. I cannot understand the point of view that denies the _overwhelming_ progress that has taken place in AI. And to deny GPT-3’s mega advance — we now have a system that can achieve state of the art or near state of the art requiring a negligible number of training cases (i.e., few shot learning) — seems wrong to me.
@hn_90b1f4
27 days ago
I think few-shot learning (or priming) is actually the main selling point of GPT-3 for most practical applications (rather than merely entertaining language generation). So if there is a method that achieves the same goal with a model that is simple enough to be used by normal developers and researchers without OpenAI-scale infrastructure, that does seem buzz-worthy.
@hn_75034c
29 days ago
This is a use case where AI-powered SQL is a solution in search of a problem, and introduces more issues than just doing boring SQL. For data analysis, it's much more important to be accurate than fast, and this article is unclear how many attempts each example query took. GPT-3 does not always output coherent output (even with good prompts), and since not 100% of the output is valid SQL the QA and risk tolerance of bad output affects the economics. OpenAI's GPT-3 API is expensive enough (especially with heavy prompt engineering) that the time saved may not outweigh the cost, particularly if the output is not 100% accurate.
@hn_73e45e
about 1 month ago
Forget AI democritization, GPT-3 is AI demoguerization. GPT-3 is singular; it is one model, one dataset, one training. Yet it will be the only one that will exist for quite some time (or by far the most available), and now it will underwrite productization and malfeasance, a la mode pay to play. For example, I recently read a paper supposedly written by a Chinese dissident virologist, which report was disseminated by a group with questionable membership. Most of the jargon in the report going over my head, I had to wonder if the otherwise convincing verbiage wasn't the work of GPT-3.
@hn_9df7b0
about 1 month ago
The main interesting thing about GPT-3 is that prompt engineering turns it into an incredibly general tool. This summary was likely generated with a prompt something like: Here's an article about x: <text of sample article> A quick summary of the article, focusing on the main relevant points and keeping critical detail: <sample summary> as an example, then duplicating that with the real article and text. That's the "few shot learning" from the original paper. It could also be that it's good enough at summarizing specifically that you don't need an example, just the right framing prompt around the article text. Either way, that kind of prompt engineering is how you get "text completion" to perform basically any text processing task, generate code or play tic tac toe, so on