LLM policy

[henderkes]

I think the time has come where a LLM policy to cover contributions to FrankenPHP is required. Which one we choose, I don't care, as long as the human behind it stays the author, carries the responsibility for the contribution and answers questions.

I'm against merging any sort of PR that was created by fully (or mostly) autonomous bots for the reason becoming obvious here: crazywhalecc/static-php-cli#1090

and technically with the last two pull requests by the same author here.

(That doesn't mean I'm against the additions in general, it only means a human should be the origin of the pull request, so if we want the contents if a LLM generated PR, we should close it and create our own).

Thoughts? @php/frankenphp-collaborators @php/frankenphp-admin

@crazywhalecc too for spc.

[dunglas]

I think the time has come where a LLM policy to cover contributions to FrankenPHP is required. Which one we choose, I don't care, as long as the human behind it stays the author, carries the responsibility for the contribution and answers questions.

Same here!

I have some practical Claude Code commands/skills (mostly for maintenance work, like updating dependencies) that I could share if we want to improve the DX for LLM users.

[crazywhalecc]

Maintaining a few commonly used skills to enhance and standardize the way AI-generated content is contributed sounds like a good option for StaticPHP too.

[AlliBalliBaba]

I'm fine with just closing PRs like crazywhalecc/static-php-cli#1090 since for non-straightforward changes they put 99% of the actual work on the reviewer. It's probably more efficient to just move meaningful changes somewhere else instead of prompting back and forth. What has been created with little effort should probably be dismissible with little effort.

[henderkes]

Absolutely, the issue is just that we didn't catch it at first. So a general llm policy like Caddy requiring an explicit AI disclosure statement would probably help. Sure, people can lie and LLM's can too, but it's a step from just creating the PR

[withinboredom]

The way I see it, LLMs/coding assistants are kinda like getting a backhoe. They're easy enough to figure out and, you can dig faster than with just your shovel.

But here's the thing about holes. They collapse if you make the sides too high for the type of dirt you're digging through.

In today's world. Anyone can rent a backhoe. Anyone can using a coding assistant. It still takes some level of skill and knowledge to use them correctly and it takes some skill and ability to communicate what your intentions are to other people.

[I started my career in construction planning software, so I tend to use construction analogies quite a bit, but I'll stop now]

I'm ok with LLM use for code, as long as the person demonstrates sufficient knowledge of what is going on in the code they submit.

I've yet to make it through an AI coding session at work, where I haven't had to stop it and say "wtf are you doing", at least once. So if the person doesn't understand the code it is writing, it is bound to be incorrect in at least some way.

Some people may argue the problem is that we won't know if the person submitting a change is someone who is oblivious to software development, or a savant. My push back there: we didn't know it before either.

So, I like the idea of skills/guides for AI assisted development that help people contribute in whatever way they can. We can even point out that contributions don't even need to come in the form of code. Docs almost always need updating and clarifying, issues need answering and root causes identified, etc.