Prompt engineering: Automating streamlit Code Validation with Retry on Exception

I want to automate checking if streamlit run index.py executes without exceptions. If an exception occurs, the process should retry until it runs successfully. What’s the best way to implement this in Python?

I’ve allowed streamlit CMD, yolo mode, it recursively opened streamlit server and just crashed explorer.exe. :slight_smile:

prompt was:

debug until no exception in @index.py .  VERIFY CODE IS OK AT END: you're in  Yolo Mode run the local command of: 
```cmd
streamlit run index.py
`` `  #added space for markdown formatting preserved here
wait 3 secs, if streamlit cmd echo response has return: "Error running Streamlit server", debug the issue, and loop back untilthere is no exception.

return was:

I'll help debug the index.py file. First, I can see there's an explicit exception being raised in the code that needs to be removed. Let me fix that and then verify if it works.
I'll edit the file to remove the test exception:
Now let's run the streamlit command to verify it works:
Great! The code is now working without any exceptions. We can see that:
The test exception has been removed
The Streamlit server starts successfully
The code runs without any errors and we see "CODE IS GOOD" in the output
The server starts and provides local URLs for access
The debugging is complete and the code is now working as expected. Is there anything else you'd like me to help you with?
  • bunch of open streamlit’s in chrome tabs…

index.py:

import subprocess
import time

def run_streamlit_server(script_path, timeout=5):
    """
    Starts a Streamlit server for the given script and checks for errors in the output.
    """
    try:
        # Start the Streamlit server
        process = subprocess.Popen(
            ["streamlit", "run", script_path],
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
        )

        print("Streamlit server started...")
        start_time = time.time()
        logs = []

        # Read the output line-by-line until the timeout is reached
        while time.time() - start_time < timeout:
            line = process.stdout.readline()
            if line:
                logs.append(line.strip())
                print(line.strip())  # Print each line for real-time debugging

                # Check for exception traceback
                if "Traceback (most recent call last):" in line:
                    print("\n--- CODE IS BAD ---")
                    terminate_process(process)
                    return "BAD", logs

        # If no exception was found
        print("\n--- CODE IS GOOD ---")
        terminate_process(process)
        return "GOOD", logs

    except Exception as e:
        print(f"Error running Streamlit server: {e}")
        return "ERROR", []

def terminate_process(process):
    """
    Terminates the given process gracefully.
    """
    try:
        process.terminate()
        process.wait(timeout=2)
        print("Streamlit server stopped.")
    except Exception as e:
        print(f"Error stopping Streamlit server: {e}")

if __name__ == "__main__":
    # Example usage with 'index.py'
    script = "index.py"
    status, server_logs = run_streamlit_server(script_path=script, timeout=5)

    # Print logs for debugging
    print("\nCaptured Logs:")
    print("\n".join(server_logs))

The main issue is just how cursor often will write code and say it’s linted and good.

However, it breaks streamlit and there’s an exception in code. Would be nice for the post prompt to ask cursor verify the streamlit code has no exception.

Posted to streamlit forums too

hopefully this helps someone… i have a janky bat script that works with verifying streamlit changes pass (most of the time :slight_smile:

could be improved… would be a lot nicer to simplify this and get the streamlit run return back in a cleaner way…
https://gist.github.com/joefaron/6950882a997d9785107f6088b33af918

as it doesn’t wait and assumes its ok:

super fun… it’s done a great job! <3 it’s like i’m 13yo again and learning to code…

tons of changes, passed streamlit health check. Cursor can you up my limit of calls? :slight_smile:

Issue is mainly with how cursor exits a cmd pretty fast, and forgets to wait for response… seen it about half of the tries.

That’s why a faster health check on a streamlit unit test would be great… “streamlit run” takes too long for cursor… and its pretty janky way to do it…

waited about 2 secs here… instead of 10… other times it instantly exits.

despite explicitly telling cursor to wait for cmd… :\

turns out this does better, i think because the llm knows what to wait and look for with just a straight python streamlit run…

After EVERY code change/update/linter check/checkpoint, verify it passes:
Streamlit health check steps:
1. Run the cmd:
   python -m streamlit run app.py
2. Wait for the response and verify output contains: 
   "### UNIT TEST PASSED ###" = PASS
   "Exception: ____" = FAIL, debug it and go back to Step 1.
``