Merge branch 'OthersideAI:main' into installer-script

Author: centopw

.gitignore

@@ -165,7 +165,10 @@ cython_debug/
 screenshot.png
 screenshot_with_grid.png
 screenshot_with_labeled_grid.png
+screenshot_mini.png
+screenshot_mini_with_grid.png
 grid_screenshot.png
 grid_reflection_screenshot.png
 reflection_screenshot.png
-summary_screenshot.png
+summary_screenshot.png
+operate/screenshots/

CONTRIBUTING.md

@@ -0,0 +1,25 @@
+# Contributing
+We appreciate your contributions!
+
+## Process
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+
+## Contribution Ideas
+- **Remove necessity for `pip install .`**: I think by uploading packages to PyPi we can reduce the installation code steps by consolidating `pip install -r requirements.txt` and `pip install .`. If that's possible that'd be great. 
+- **Improve performance by finding optimal screenshot grid**: A primary element of the framework is that it overlays a percentage grid on the screenshot which GPT-4v uses to estimate click locations. If someone is able to find the optimal grid and some evaluation metrics to confirm it is an improvement on the current method then we will merge that PR. 
+- **Improve the `SUMMARY_PROMPT`**
+- **Create an evaluation system**
+- **Improve Linux and Windows compatibility**: There are still some issues with Linux and Windows compatibility. PRs to fix the issues are encouraged. 
+- **Enabling New Mouse Capabilities**: (drag, hover, etc.)
+- **Adding New Multimodal Models**: Integration of new multimodal models is welcomed. If you have a specific model in mind that you believe would be a valuable addition, please feel free to integrate it and submit a PR.
+- **Framework Architecture Improvements**: Think you can enhance the framework architecture described in the intro? We welcome suggestions and PRs.
+
+## Guidelines
+This will primarily be a [Software 2.0](https://karpathy.medium.com/software-2-0-a64152b37c35) project. For this reason: 
+
+- Let's try to hold off refactors into separate files until `main.py` is more than 1000 lines
+

README.md

@@ -46,8 +46,15 @@ git clone https://github.com/OthersideAI/self-operating-computer.git
 cd self-operating-computer
 ```
 
+3. **Create a Python virtual environment with Poetry**.
+
+```
+cat requirements.txt | xargs poetry add
+```
+
 3. **Create a Python virtual environment**. [Learn more about Python virtual environment](https://docs.python.org/3/library/venv.html).
 
+
 ```
 python3 -m venv venv
 ```
@@ -82,13 +89,13 @@ operate
   <img src="https://github.com/OthersideAI/self-operating-computer/blob/main/readme/terminal-access-2.png" width="300"  style="margin: 10px;"/>
 </div>
 
-### Contributions are Welcomed! Some Ideas: 
-- **Prompt Improvements**: Noticed any areas for prompt improvements? Feel free to make suggestions or submit a pull request (PR). 
-- **Enabling New Mouse Capabilities** (drag, hover, etc.)
-- **Adding New Multimodal Models**: Integration of new multimodal models is welcomed. If you have a specific model in mind that you believe would be a valuable addition, please feel free to integrate it and submit a PR.
-- **Framework Architecture Improvements**: Think you can enhance the framework architecture described in the intro? We welcome suggestions and PRs.
+### Contributions are Welcomed!:
 
-For any input on improving this project, feel free to reach out to [Josh](https://twitter.com/josh_bickett) on Twitter.
+If you want to contribute yourself, see [CONTRIBUTING.md](https://github.com/OthersideAI/self-operating-computer/blob/main/CONTRIBUTING.md).
+
+### Feedback
+
+For any input on improving this project, feel free to reach out to [Josh](https://twitter.com/josh_bickett) on Twitter. 
 
 ### Follow HyperWriteAI for More Updates
 
@@ -97,4 +104,8 @@ Stay updated with the latest developments:
 - Follow HyperWriteAI on [LinkedIn](https://www.linkedin.com/company/othersideai/).
 
 ### Compatibility
-- This project is compatible with Mac OS, Windows, and Linux (with X server installed).
+- This project is compatible with Mac OS, Windows, and Linux (with X server installed).
+
+### Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=OthersideAI/self-operating-computer&type=Timeline)](https://star-history.com/#OthersideAI/self-operating-computer&Timeline)