What if we used AI to extract information from The Crucible and add it to pod.hubbe.net?
It would still need manual review to make sure it’s correct, but it could potentially help document a lot of dark corners which you otherwise have to search The Crucible for.
2 Likes
I think this is a truly beneficial idea and right in the wheelhouse of LLMs. One important thing to plan around is the cross version information that sits side by side without always explicitly naming the distinction. An agent pointed at the Crucible could produce confident and well formed documentation that is no longer true in the latest version. It should be explicit about where it does and doesn’t apply. In my day job this has been a major challenge for us that leads to reviewer fatigue when they continually encounter things that used to be true but aren’t anymore. However, this isn’t a barrier so much as a design consideration. Providing the source code as context allows for verification and triage prior to human review.
2 Likes
I think if we focus on documenting useful things, and not every little detail, the chance of things getting outdated is pretty small. Also, ProffieOS strives to be backwards compatible, so old documentation should still be valid, in most cases.
Friends… I think I can be of service here. We want to build an agent for lightsaberzzz?
I have no interest in creating an agent. My goal is to improve the documentation.
Interesting. An agent isn’t necessary.
What do you mean by “improve”?
Where would you start? As an example, I’d be interested in a complete list of defines with their OS implementation history and a usage description.
There are other topics that could be brought in, like power distribution techniques.
There’s also alot of “how tos” and the really interesting stuff where people are building new things and getting help with code that could turn into an “implementations” section.
I dunno, it seems like a good idea. There is a ton of valuable info at the Crucible that could use a home in the POD.
Personally, I’ve avoided AI but watched it change my wife’s career for the better. She got garbage output until using Claude for her purposes, and her usage budget has been increased 10 fold based on successful implementations. So which AI you choose may also matter.
Yes, useful topics and limited to a focused POC (proof of concept) on 2-3 of them. I am completely aligned that a persistent agent is in no way the goal and I’d push back on doing that too. On the other hand, initiating some agentic AI work (just meaning that is how you set parameters for gathering and compiling information on the topic/s with necessary contextual awareness) can be a low stakes way of seeing if there is value in doing this. One of the most interesting things might be some investigative work to gather trends and common topics to see how it aligns with what would be our gut feel.
I think you’re right about the backwards compatibility, though that same principle would apply to other areas where prop varies or the answer isn’t directly clear after a long thread. Not something that would stop the work, but based on my experience it would generate confidently wrong answers if not accounted for in context and parameters of the AI task. @profezzorn did you have a couple topics in mind?
Make it more complete and easier to use.
In this particular case, I think it’s just about taking information from the forum and putting in a better, more easy-to-find location. (While also cleaning it up a bit to make it easier to read.)
This page contains all supported defines:
there is literally a “howto” section on the pod site, maybe it should go there?
I mean, if people would do it, that would be even better. The vast majority of the information on pod.hubbe.net comes from me, but pod.hubbe.net is basically a wiki. Anybody can hit the edit button and make changes. (which I will then review before they go up on the site.)
Not specifically. But there are some long posts written by me that could definitely make good additions to the POD. One way to do this would be to have a tag, then a script that finds tagged posts and tries to make them into POD pages.
Yes, this seems like a good way to add value from known good information. How can I help? Happy to take a couple of them as the proof of concept.
1 Like
I think the whole idea is great!
The company I worked for purchased an enterprise account for everyone with chatgpt. I reluctantly started using it and it has been surprisingly useful.
Even if I have to fight my internal “ai bad” thoughts.
1 Like
Why not prop specific defines? Bridging that gap makes sense to me.
Good plan.
Your AI could be promted to look for word count, content type, and even who you were speaking too. So any thing between you and the original high tier contributor/creators like Shtoky, Fett, NoSloppy, Sabersense…could be identified and prioritized.
One topic in particular that I’ve looped back to lately is what Sabersense called his “array selector” and then Fett defined as FETT263_MANUAL_BLADE_ARRAY.
Anyway, I think this use for AI follows the Crucible motto: “It has to make sense.”
And I suppose I do use AI as it is everywhere. But in my work, which is done in a 1955 gymnasium in which I have the authority refuse screens (The Battle of Smartboard ranks up there with the Chocolate Milk Wars and the Great Pickleball Conflict) it has no way to encroach.
Props are kind of separate from ProffieOS from a principle standpoint. They’re not backwards compatible across their versions in the way ProffieOS tries to be, and have their own documentation in the header files, which likewise, at least for the two that have been around and have been updated (BC and Fett), tends to change between versions.
I do believe there are at least a few pages on the POD for Fett’s prop.
I don’t think anything prop specific belongs in at least the POD’s main config documentation for those reasons, probably in their own pages (but organization might be messy for the above reasons…) if anything.
1 Like
On that front, I suppose a primary concern would also be to make sure things aren’t duplicated, and that they’re organized reasonably with what already exists (I’m sure there’s plenty of unique things on the Crucible that’d be fine entirely relegated to their own isolated page, but..).
If you follow through with it I’ll be curious I guess. It seems like it’d still be a non-trivial amount of work to setup, review, and curate, but I guess it depends on what quality you’re after.
If anything, scraping the Crucible to get a “bulleted” list of items/posts that may be worth someone taking the time to formalize into a POD page would maybe be better (as opposed to having an LLM synthesize it outright). I’m not sure such a person exists, but if they did: it could also maybe make it easy for someone who wanted to find something to work on…
There should probably be pages for each prop that lists their defines, with links from this page. The defines should be relatively easy to extract form the prop files themselves.
1 Like
That sounds like a good prompt for the droid 
1 Like
In my experience, AI is really good for this usecase.
You can of course start small like providing a list of topics that are worth reading like @ryryog25 said. And later expand it to update pages automatically (with or without review in the future).
With the right set of instructions this is easy to do. Funny enough, creating the right set of instructions is very much doable with AI 
I just learned discourse even has an MCP available, so AI interaction/integration is now supported “out of the box” without having to scrape the website itself.
1 Like
Get a team of real people to do this.
Depending on how it’s used, it’s actually really useful. I get the hesitation, but i’ve been using it extensively (mainly in my work) for the past year. I’ve been blown away on what it can do and how it made ‘my life easier’ so to speak.
In my 6 years of working as a software engineer I’ve never liked writing new or updating documentation, so I’m glad I can hand it of towards AI so I can focus on the stuff I like to do.
I think extracting useful information from a forum is kind of a pain to do and consumes a lot of time. This way a lot of work can be done automatically 
Then again, that’s how I feel, and I’m not trying to force anyone on using AI. I’m just saying, give it a shot
I tend to agree.
Less documentation is better than more if what’s there has proper effort put into it, IMO.