After thoroughly testing the new Llama 4 Scout and Llama 4 Maverick models, the results are disappointing for creative writing tasks. Despite the impressive technical specifications, such as the 10 million context window for the Scout model, the actual performance falls short.

The log lines and story outlines generated by both models lack coherence and fail to present compelling narrative ideas. The outlines often mix up the order of events and introduce plot elements without proper foreshadowing, indicating a poor understanding of narrative structure.

The prose generated by the models is also lackluster, relying heavily on telling rather than showing, and lacking in character depth and emotional resonance. The models struggle to follow specific prompts and instructions, often introducing unrelated elements or continuing past the requested scope.

While the models perform slightly better on the ad headline prompts, the results are still not significantly better than those from other, less powerful models. The email newsletter prompt also produces generic and philosophically-inclined content, rather than the concise and personable style requested.

Overall, the Llama 4 models do not demonstrate a substantial improvement over their predecessors for creative writing tasks. The models' technical capabilities do not translate into meaningful narrative and prose generation, leaving much to be desired for writers and storytellers. Alternatives like DeepSEE or fine-tuned versions of the models may prove more useful for creative applications.

The announcement of the new Llama 4 models, Maverick and Scout, has been met with some controversy and skepticism. While Meta claimed impressive capabilities, such as the 10 million context window for the Scout model, there are indications that these claims may not accurately reflect the real-world performance of the models.

Several experts and researchers have tested the Llama 4 models and found that the large context window does not necessarily translate to better information retrieval or overall performance. The ability to hold a large amount of context does not guarantee that the model can effectively utilize that information, and in some cases, it may even hinder the model's ability to focus on the task at hand.

Furthermore, there are concerns that Meta may have inflated the performance comparisons of the Llama 4 models against other benchmarks, such as Gemini 2.0, Flash Deepsee V 3.1, and GPD40. It appears that Meta may have used slightly fine-tuned versions of the Llama 4 models for these comparisons, which could have skewed the results in their favor. This practice, while not entirely uncommon in the industry, raises questions about the transparency and accuracy of the claims made by Meta.

The author's own testing of the Llama 4 Scout and Maverick models for creative writing tasks has not yielded particularly groundbreaking results. The generated log lines, outlines, and prose samples demonstrate a tendency towards generic, unspecific ideas and a lack of coherence in storytelling elements, such as proper foreshadowing and pacing.

Overall, the findings suggest that while the Llama 4 models may have some improvements over their predecessors, they do not appear to be the revolutionary advancements that Meta has claimed. The author recommends relying on more established and well-tested models, such as Sonnet 3.7 and Gemini 2.5 Pro, for creative writing tasks, as they have demonstrated more consistent and reliable performance in the author's own testing.

Both Llama 4 Scout and Maverick models struggled with the outlining prompts. The prologue sections provided by the models were more akin to a series of cinematic shots rather than an actual story prologue. The chapter summaries lacked cohesion, with Llama 4 Scout even mixing up the order of the chapters.

While the models were able to generate a full outline, the quality was lacking. The chapter summaries were often generic and did not provide any real plot or character development. Additionally, the Llama 4 Maverick model introduced new story elements that were not part of the original prompt, demonstrating a lack of understanding of the context provided.

Overall, the outlining capabilities of both Llama 4 Scout and Maverick were disappointing. The models were unable to craft a coherent and well-structured outline, which is an essential skill for creative writing. These results suggest that the models still have room for improvement when it comes to long-form narrative generation and maintaining narrative coherence.

The Llama 4 Scout and Llama 4 Maverick models from Meta have been met with mixed reviews. While they boast impressive technical specifications, such as the 10 million context window for the Scout model, the actual quality of the generated prose appears to be lacking.

The log lines and story outlines produced by these models are described as generic and lacking in specific plot details or compelling narratives. The outlines also exhibit issues with maintaining coherent story structure and sequence of events.

When it comes to the prose generation prompts, the output is characterized as "telling" rather than "showing", with an overuse of adjectives and adverbs instead of immersing the reader in the character's perspective. The models also struggle to follow specific instructions and tend to introduce unrelated story elements.

However, the models do seem to perform better on the ad headline prompts, producing usable and reasonably compelling headlines, likely due to Meta's access to a wealth of high-performing ad copy data.

Overall, the assessment suggests that while the Llama 4 models have technical capabilities, they fall short in terms of generating high-quality, coherent, and compelling prose for creative writing tasks. The models may require further fine-tuning and development to truly excel in this area.

The ad headline generation prompts seem to be the bright spot for the Llama 4 models. While the models struggled with other creative writing tasks like outlining and prose generation, the ad headlines produced were surprisingly decent.

For both the Llama 4 Scout and Llama 4 Maverick models, the ad headlines showed more coherence and creativity compared to the other prompts. Some examples include:

• "What if everything you knew was a lie? The hunger is coming. What happens when the very creatures that will eat you are not the only monsters lurking in the shadows?"
• "The city that lies. What secrets is your sheltered life hiding? Uncover the truth in a gripping fantasy adventure."
• "Three races, one goal: your demise. Can Lethan outwit the Sucrine, Caraggoth and Zolath to uncover the truth?"

These headlines capture attention, tease the reader with intriguing questions, and hint at the core conflict of a fantasy story. While not groundbreaking, they demonstrate a level of coherence and marketing savvy that was lacking in the other creative writing tasks.

The email newsletter prompt also produced reasonably usable results, though it struggled to fully capture the concise, philosophical tone of Ryan Holiday's writing that was specified in the prompt.

Overall, the ad headline generation appears to be a relative strength for the Llama 4 models, suggesting that Meta's access to large datasets of high-performing ad copy may have paid off in this specific area of creative writing. However, the models still fall short compared to top-tier options for other creative tasks.

In summary, the Llama 4 Scout and Llama 4 Maverick models from Meta have not lived up to the hype. The models struggled with various prompts, including creative writing tasks, outlining, and generating coherent prose.

The key issues identified include:

• Lack of groundbreaking or compelling story ideas in the logline and outline prompts
• Difficulty maintaining context and logical flow in the outlines, with chapters out of order
• Overly "telling" rather than "showing" in the prose prompts, with excessive use of adjectives and adverbs
• Tendency to introduce unrelated story elements when prompted for a specific scene
• Disappointing performance on the longer-form article prompt, with word counts falling short of the target

While the models may have some potential for fine-tuning and further development, they do not currently measure up to the capabilities of other leading language models, such as Anthropic's Sonnet or Anthropic's Gemini, for creative writing tasks. Prospective users may be better served by exploring alternative options, particularly if they have a limited budget.