Summary of Key Points
This article presents a real-world test that involved three major Chinese open-source large models (GLM5.2, Kimi K3, and Qwen3.8-Max) collaborating to transform a poorly maintained website (a “messy codebase”). The test assessed their performance across six tasks: understanding the project’s current state, fixing actual bugs, modifying functionality, designing user interfaces, and handling complex backend processes. The results show that no single model dominated throughout the entire process. However, the capabilities of these Chinese open-source models have far surpassed previous predictions that they would take 6 to 12 months to catch up with foreign models. Each model exhibited its own strengths and weaknesses, collectively demonstrating their ability to handle complex real-world development challenges.
Detailed Breakdown and Interpretation
1. **Task 1: Who Can Quickly Understand the Current Situation?**
Test Task: The models were asked to analyze the current state of the semi-finished website, distinguishing between completed features and plans outlined in old documents to avoid redundant work.
Results:
- Qwen was the fastest and most accurate (completed the task in less than 10 seconds, accurately identifying the most recent activities).
- Kimi acted as a mediator, providing concise information but made careless mistakes in counting data.
- GLM understood the underlying structure of the website (e.g., identifying 14 CMS systems), but mistook old plans from April for current requirements, resulting in wasted effort.
Popular Explanation: This task is like taking over a house that’s halfway through renovation. Qwen is like a supervisor who constantly monitors the progress; GLM is like an experienced builder who refers to old blueprints; Kimi is like a quick-guiding mediator whose explanation is helpful but lacks detail. Understanding the current situation is crucial as it sets the foundation for any subsequent actions.
2. **Fixing Actual Problems: Who Can Actually Fix Bugs and Modify Functions?**
Test Tasks:
- Resolve an error caused by the front-end not having the CMS enabled.
- Modify the colors and animations of a carousel and the “latest insights” section.
Results:
- Error Resolution: Qwen quickly resolved the issue by enabling the CMS (most efficient).
- GLM took longer but reused Qwen’s solution (showing smart problem-solving).
- Kimi only provided a plan without making any actual changes (failed the task).
- Carousel: GLM offered the most comprehensive features (automatic playback, drag-and-drop, and seamless looping), but there were minor issues with the animation.
- Color Modification: Qwen’s solution was the most complete, although it missed the hover effect; GLM only modified the parts that were explicitly requested; Kimi made almost no changes (failed to understand the task fully).
Popular Explanation: This task is like fixing a piece of household equipment. Qwen took direct action and solved the problem efficiently; GLM was slow but flexible in finding a workaround; Kimi offered suggestions without actually making any improvements.
3. **Aesthetic Competition: Whose Home Page Design Matches the “Tech Media” Brand Best?**
Test Task: The models were asked to redesign the website’s Hero section, keeping the content intact while changing the layout, style, and animations to convey a professional and cutting-edge feel.
Results:
- Kimi: A minimalist tech-style design using blue colors and geometric animations (won, as it captured the essence of being an AI-related brand).
- Qwen: A modern media-style design with large headlines and a minimal background (came in second, though it lacked a strong tech vibe).
- GLM: A magazine-like design with italic fonts and editorial elements (came in third, as it felt more like a magazine cover than a tech platform).
Popular Explanation: This task is like designing a storefront for a tech brand. Kimi understood the brand’s focus on innovation and created a design that reflected this; Qwen designed a simple layout suitable for a general media outlet; GLM’s design was more fitting for a fashion magazine, though it didn’t fully convey the tech vibe.
4. **Complex Process Challenge: Who Can Improve the Backend Review System?**
Test Task: Optimize the CMS review workflow to display pending articles and expand details on the right side without requiring page jumps.
Results:
- Qwen: Created the most user-friendly interface, with all necessary information readily available.
- Kimi: Completed the task quickly but simplified the preview of the main content (required additional page jumps).
- GLM: Had significant issues, as the image loading in the main content area didn’t work properly.
Popular Explanation: This task is like streamlining an office process. Qwen made the review process smoother; Kimi improved it but didn’t completely solve the problem; GLM’s changes made the system more complicated. Qwen demonstrated a better ability to handle multiple interconnected tasks efficiently.
5. **Final Conclusion: Chinese Open-Source Models Are Competent, But with Different Strengths**
Strengths and Weaknesses of Each Model:
- Qwen: Fast and accurate, but sometimes lacks thoroughness (e.g., using clever tricks to simulate loops).
- GLM: Stable and creative, but slow and easily distracted by old data.
- Kimi: Innovative and fast, but careless with details and prone to missing important modifications.
Key Conclusion: Previous predictions suggested that Chinese models would take 12 months to catch up with foreign ones. However, this test shows that they can handle complex projects like this one effectively. Although not perfect, their progress is evident, and they are now capable of delivering meaningful results.
This experiment demonstrates that Chinese large models are not just theoretical concepts; they can actually be used in practical applications. Each model has its unique strengths, and when combined, they can solve many real-world problems. For ordinary users, there’s no need to rely solely on foreign models—Chinese open-source models are now ready for use in various scenarios.