Out of morbid curiosity, I decided to do a comparison of composer in 0.39.6 vs 0.40.4, so I did a side by side comparison. Granted, this is hardly a comprehensive test, and also realizing you can’t get the same exact result from the same prompt each time, but with the small test I ran, I found some interesting things.
I have to admit, I did go into this having a bias thinking that 0.40.4 might be worse. It wasn’t, actually, it was the opposite. Here’s what I found off the bat (again, granted, this was just one small test case, but I had a much smoother experience with 0.40.4). Model was claude-3.5-sonnet for both.
I actually recorded a video of the entire run but I need to go back and cut through it since it’s a long post and some stuff needs to be redacted.
The initial prompt I used was:
From the attached screenshot, and using vanilla HTML, CSS, Javascript, and UIkit, create a working single landing page that looks like this. Specs are as follows:
1) The name of this page is called “Sponsorship Opportunities”, which is broken down into 3 distinct sections.
- The first section has copy on the left, with an image on the right.
- The 2nd section has an image on the left, with an accordion widget on the right. The accordion contains items that should expand when clicked on, which also changes the image on the left accordingly.
- The 3rd section is a photo gallery, spanning the full screen (edge-to-edge / no container), of 8 photos. Hovering over any photo reveals, in the top right hand corner, an icon indicating the photo can be expanded. When clicked on, the photo appears in a lightbox overlay, with navigation to move between photos.
2) The red accent color is #dc291e
3) Fonts are Helvetica / Arial / sans-serif.
4) Use picsum for all placeholder images, for now.
Here’s what I noted.
- Composer 0.39.6 can see images, but 0.40.4 can’t yet, and even then, 0.40.4 pretty much nailed my prompt in one shot correctly. 0.39.6 completely failed in generating the photo gallery portion which resulted in several more prompts to iterate over and correct.
-
I had one instance of prompting 0.39.6 where it gave me the code instead of applying it. I asked it to apply, and it did. I had no instances of that in 0.40.4.
-
When iterating over the photo gallery, 0.39.6 took numerous prompts to get it to understand the layout I was describing (even though it also had the screenshot). 0.40.4 understood the layout I was describing in one shot.
-
0.39.6 created a single file, whereas, 0.40.4 created multiple files and linked them together (in context, this was preferred)
-
Both of them failed to populate the initial index.html file off the initial prompt (they both created the file, but they were empty initially. I had to copy and paste the contents afterward). 0.40.4 created multiple files (a .js and .css file), and those were populated initially, correctly.
I’ll think up more tests to run, but thought that aside from 0.40.4’s composer not being able to see images, it’s certainly not degraded in quality that I can see. Kudos to the Cursor team.