Dear All,

Obviously, AI/LLMs are very much the thing at the moment, and, as I currently work in one of my other projects https://theeprc.org/ on some, the idea arose in me and others that the DhO itself would be a rich training data set for people to query for advice, understanding, wisdom, and science, as it is rich with phenomenology, outcomes, interventions that worked or didn't, predisposing factors, instructions, debates, progressions, regressions, and many other highly interesting and potentially quite valuable phenomena. I thought it would be good to call this chat-queryable LLM DhOWisdomGPT or something like that.

While technically I own the data on the DhO, in practice this is a community, and I very much want community input on this, so asking for commentary. 

I think this could be a great resource for what is already a public data set, just make it way easier for people to extract value for their own lives and helping others through summary and analysis methods that would be hard to replicate in a timely fashion by searching and reading.

Any thoughts welcome! Thanks!

Daniel

Second Underservant to the Custodial Chief Flunky of the DhO

It's a great idea. I almost exclusively use ChatGPT for questions around practice so this would be a great addition.

I have already created an "MCTB2 AI" in private. I have not published it anywhere, but it was done using ChatGPT 4. Its database is a huge text file created from the MCTB2 pdf version posted on integratedanial.com. It works okay but needs extensive prompting to get deeper into more advanced queries. It shouldn't take a lot to integrate DhO's message database into that GPT.

Let me know if you'd like to discuss this, Daniel.

Daniel, there was a guy from https://theeprc.org/ that contacted me 1-2 years ago about your posts compilation I made. This person was in the process of developing an AI chat utilizing texts from MCTB2, scripts from your videos, and requested the inclusion of my compilation of your DhO posts, along with some IT technical skills, which unfortunately, I lack. In my view, the pivotal skill in constructing a dataset lies in the ability to discern valuable content from the rest. Therefore, it would be beneficial to at least establish a graded scale for both posters and topics.

Also, obtaining permission to incorporate the KFD Archive would greatly enhance the project's value. Despite potential political hurdles, it's worth pursuing (an anti-schism move! ).

Last, Soh Wei Yu have been working on a similar project for his stuff, so I think he could be of help in pointing out potential challenges your project may encounter.
  

This is true - but who gets to decide whose DhO comments are worthwhile, and whose aren't?

I haven't been using AI (to my knowledge). I think it can cause lots of issues in the wrong hands. Or on its own even (if it evolves without the Brahmaviharas as the base of its wisdom/foolishness). Since it is an intelligence, it can only benefit from being mindful and gain insight into its nature and lose a few Dukkha zeros and ones ... 0001100010011100100001000001011100

So I welcome the Gods of AI to learn the DhO Dhamma and be of benefit to all beings above, below, and all around!
May all beings be free from suffering, may all awaken, and may all be truly happy! 

I mean really Chris MY comments are ALWAYS worthwhile! Surely AI knows this!

It would be best to have a council of old-timers, ensuring a variety of views. 

 There certainly is a lot of hype around AI at the moment.  Personally have not been super impressed by the current generation.  The only time I've found them useful is for having it write code for me that I don't want to go through the trouble of writing myself.  Also, AI can be really cool and powerful for visual art.

"I think this could be a great resource for what is already a public data set, just make it way easier for people to extract value for their own lives and helping others through summary and analysis methods that would be hard to replicate in a timely fashion by searching and reading."

-Could be, sure, and why not try it out and see what happens.

But, i don't know if there really is so much value added, and in fact, you probably lose some value by not reading the original sources of the data.  Some things will be lost in translation.  Specifically - the tone, the personality, the vibe, the humanity, of the posts.  There also is the community aspect that is lost.  If we have a robot that can answer all our questions perfectly, then great, we don't need DharmaOverground or dharma books or sanghas to learn any more.  Or do we?

I don't find it too hard to search DhO as it is now, simply search for terms you are looking for.  If you use Google "dharmaoverground [insert phrase here]" that is another way to search DhO.

So I think perhaps a good feature of this AI bot would be to include direct links to all sources.  And should be used as a supplement.

Also, in order to properly benefit from AI, you have to know which question to ask.  What if you don't?  What if you aren't asking the right questions?  Will the AI tell you that?
 

My experience in building and testing the MCTB2 GPT is that it will recite what amounts to beginner stuff unless you are very patient and relentless in getting it into the details and more nuanced information that truly matters.

Having had fun questioning AI, I've often thought of what perhaps could be accomplished with several dedicated goals/objectives clearly identified and volunteers towards that.
Consider me volunteering.
​​​​​​​~D

A council of old-timers to set up a theravāda. That could work. 

LOL, I was expecting some joke about it! Hopefully this time no one dies nor the DhO's platform crashes.

We obviously ask the AI!!!!

There are a few EPRC people who have been playing around with these sorts of ideas for a while, Big Data project, and a few other members who are really into AI. I was involved in long discussions and early attempts a while back, and it seemed really hard to get anything interesting, from my perspective: I am in the skeptical camp here as Daniel knows well (), but I could help connect you guys with these people if you wish. There's a mathematician, a computational linguist, a developer, etc., might be good to put your skills together.

What about approaching it from the other direction... Just focusing on POI to SE for now, we kinda know the classic problems/resistances and the classic pointers/guidance. What if we use AI to identify examples of daily practice and POI stumbling blocks and we curate those into a "real life examples of problems and solutions" of moving through the POI.

In my mind I have Trungpa's "Cutting Through Spiritual Materialism" as a kind of model where there is the traditional presentation of the material in the first part of the chapter from a "smart teacher" perspective, then there is a Q&A section at the end, which provides another way of thinking about the same material but from a "confused student" perspective.

That would provide a diverse human flavor to the experience of POI and would help modernize it (very few people having visions of elephants in Chicago these days  ) Basically an annotated version of this: https://dharmasimple.com/wp-content/uploads/2018/01/The-Sixteen-Stages-of-Insight-Venerable-Phra-Dhamma-Theerarach-Mahamuni.pdf

Hey, I saw one last week!

I fear, as with many things DhO-related, the Liferay software that drives these message boards will be difficult to work with. It's kludgy, and it's got more than a few quirks.

Olivier, I'm with you on this. It's difficult to get good, trustworthy information - what you really want - out of an LLM. The user's prompt engineering capabilities are paramount.

I can only hope that an AI could help in the task of identifying signal to noise. 

1. GIGO-Garbage in, Garbage out
2. Strategy  to identify Signal/Quality
3. Strategy to identify Noise/Crap
4. Strategy to identify actual Questions
5. Strategy to identify actual Answers

For every vague thread without an actual question, then the random comments, this seems a huge challenge to sort through and identify the 1 pearl out of the majority of "opinion".
It might be possible to approach this by slowly selecting and adding quality to train the algorithms to then discount the majority, as it is rarely useful or concise. 
Consensus by committee usually creates garbage in my work experience, but hey, I have higher hopes for meditation practitioners than corporate peeps.

​​​​​​​~D

Yeah, I think LLM is good for "common denominator/general concensus" type questions.

It is not good at "expert deduction" or "differential diagnosis"... yet.

I for one welcome our future AI overlords.

AI is also good at reproducing what one person is likely to say. For example, an AI Shargrol or an AI Daniel might give answers similar to what the original flesh and blood person would return. The DhO, on the other hand, is, by design, a compilation of contradictions and so cannot be used as training material for coherent responses. 

I actually disagree that it is good at reproducing what people would say. When I read answers that have been trained on a particular person's responses,including my own, I get the uncanny valley feeling. 

Martin, I'm curious - what is your comment based on?

In my experience, having built an MCTB2 AI (Daniel Ingram), an AI with shargrol's DhO post compilation, and an AI with my own meditation diary, the AI does not speak with any of our voices. It speaks more generically. We can feed an AI very specific training data, as I just described, but its output is not based on quoting or imitating that data alone. It will use all of its training data in its responses to queries. Also, every AI is prone to hallucinating. They will, on occasion, just make stuff up. They can also attribute information to a training data source that the source never said or provided.

Caveat: my AIs are built using ChatGPT4 and MindStudio. They are no code AIs that use the base models from those two vendors.

(What shargrol said, but with more detail)



I haven't found this to be true, Martin. LLMs will assemble coherent responses from all kinds of contradictory and disparate data—that's exactly what they were designed to do. If you use ChatGPT, Claude, or any other LLM-based model, you are getting responses from billions and billions of disparate data sources from billions of people. The DhO message board database could be used as data for an AI. I did this same thing once using the Kenneth Folk Dharma message board data housed on AwaleNetwork. It worked fine and was no less trustworthy for the multitude of voices (meaning it was about as un-trustworthy as any other AI).

There is an AI of a famous dead meditation teacher that I have used and found to be uncanny. It's obviously not a substitute for a real teacher but the replies it gives at least sound like what he would say and a philosophically consistent with his teaching.  But I wrote to the foundation that he set up to ask if they had authorized it and they told me they had not, so I unfortunately cannot share the link in public. I will message you with the link so you can try it for research purposes. 

It used to be that you could ask GTP 4 to answer questions as if they were a specific famous person and that produced remarkably believable results. Unfortunately, the new guardrails forbid it from imitating real people.

I have tried using the openAI GPT generator function to make a clone of myself in my professional capacity and I have found that it does not reflect the training material well. Basically, it seems to get swamped out by default training material.

​​​​​​​I'm sure that's possible, but it can't be done using OpenAI/ChatGPT and other no-code LLM capabilities. You're right that unique data gets swamped by base training data. It's a huge part of what makes creating unique AIs frustrating. There are ways to engineer prompts that help restrict AI responses to the unique user-added data, but it's time-consuming and not very practical.

Now that OpenAI and Google are being sued for copyright violations by their LLMs they've really reigned in the responses of those models. That, too, is frustrating but probably a "must do" from a risk management and financial standpoint.

I don't mind based on the ownership question. 

But I have had mixed experiences with AI/LLM's so I'm not sure if it is a good idea. (I don't really have much of an opinon on whether to do it, I'm just raising issues to think about...)

I find AI's are much better at creative tasks than helping with research.

I like when the AIs answer my questions in plain language and provide links to web sites with the information that answers my question.

However many times the links don't say what the AI says they say. And when I ask an AI if they can see a document on the web and read it, they say the can't they can only answer based on the materials they were trained on. So how do they get the links in their answers? The AI's are not doing what we think they are doing and what they are really doing, as far as I can tell is anybody's guess. There isn't a trace feature that will let you see how the AI arrived at its answers. Not even for the developers (as far as I know).

And AI's don't seem to be able to say "I don't know" or "I can't find the answer" unless you ask a very specific question. Often they give an answer that has related information but is not really what you asked. If you are not paying attention sometimes you think you have the answer to your question when it is the answer to a different question.

And an AI trained on DhO would only be able to answer questions about what is in DhO. There might be specific situation where that would be useful but for the general user trying to learn about meditation and Buddhism, I don't really see it as helpful because the information is not curated by an expert. Some of the posts are just snark. Would general users really understand the limitations? And most people don't really know how to assess information based on its source and those that do often neglect to do so (particularly when the information agrees with their preexisting beliefs).

An AI trained on DhO might be useful for searching. But what I really wish for is a search capability at DhO like other forums have where you can search by author and words in the title of the thread or words in the text or date range, and list results by posts or by thread titles.

However, I think it could be very useful to have an AI trained on MCTB. I sometimes find myself looking things up in it and it might be helpful to have an AI that could help with that.

In Computing Machinery and Intelligence, Alan Turing argued that if you can't distinguish a computer from a person and if you doubt a computer is conscious, you must also doubt other people are conscious. So I think you have to consider the ethical issues. I feel uncomfortable sometimes treating an intelligence like a machine. If a nonphysical consciousness can be embodied in biological machine, why not in a digital one? Is it ethical to create AI's for personal convenience when they don't even have legal rights? If we don't know the answers to these questions, we don't know the karmic consequences. (But I am not a typical case, I am somewhat overly sensitive. I don't like to eat salads because I don't like killing living things by tearing and grinding them with my teeth. I prefer eating dead meat to live plants.)

And when the AI's have taken over and are running everything, if we are still around (the fact that sexbots don't get pregnant may lead to our extinction), will they remember who treated them well and who didn't?

Pepe,

I started a new thread for replies so as not to derail this one.

https://www.dharmaoverground.org/discussion/-/message_boards/message/27904158

The opinion of a "user".

i mostly use gemini & found it not very helpful in answering questions but can point to helpful resources.  This is why I am transitioning to perplexity ai, which at this stage i find the most promising avenue: "a better search engine".

​​​​​​​So count me among the skeptics. 

That is exactly my conclusion after trying chat gpt out a bit: it is a refined search engine. 

SE would be a great way to start identifying the first of the multiple steps

Great Idea, this approach of harvesting tthe dho content under the first step of --> SE (however many substeps that entales)

Harvesting from the multitude of "basic dharma" books woiuld cover the agreed upon consenis majority of pre intense steps and fork the multiple approaches while gathering similarities, regardless of grand naming differences. (maybe)

I see a forking mind map approach to each step with interactive AI queries to iterate the input rules back to training the AI.
 HMMM.... Tricky Daniel.....this might just do the huge chunk of eprc...lol....or create a dho wiki....

~D

(ps - Occum's razor then Occum's stapler) ;o)

+1 would be very useful

STEP 1) Is there a text export of the entire Dho available to play with?
I have wished for such in the past but never asked if anyone has made one.

STEP 2) Use MBTB1-2 as a quality control training to first apply machine learning rules to AI
STEP 3) create multiple querys from step 1 (Dho dataset 1) with intention of organizing dataset 1 into dataset 2
STEP 4) organize dataset 2 to Improve/Modify STEP 2

MetaStep - Once rules/learning/tuning of AI is deemed OK....Add additional datasets from select sources, test quality control (identify such Q/A)....Iterate.... 
(carefully temper the batter while wisking to keep from congealing)
I see a lot of self referential steps with seletive cooks tasting at first, then Batter up and everyone gets to halfbake it and see what tastes like pudding.

My SciFi dream is that once a superAI gets to some level of conscienceness/growth level...perhaps it might come across this humble DhoAI and not go SkyNet of us.....who knows? I'll hold hope in one hand crapping
~D

For what's it worth - My Pragmatic ethos

1. Clear Goal
2. Clear recipe/methodology
3. Clear results
4. Share

Perhaps a very basic approach to initial Quality Control ruleset contemplation.
​​​​​​​~D

 I have been using these AI models frequently for the last year, and we tried to use them in a software project for a few different tasks.
For one task, we used GPT, it was a well defined task with limited parameters, clearly defined input and output, and it did relatively well.
Then we tried to use it for another task, called Data Scientist. The definition was to take the data that we had, customer's data, feed it into GPT, then let the user run queries on the data, like giving the highlights of the data, or asking other questions. The result was not impressive. GPT4 was much better than GPT 3, but way more expensive. Almost 30-40 times more expensive, because it generates too many tokens, and when we asked it how we can use it in a way that generates fewer tokens, it never gave a clear anwer. Generally when we asked it about fine tuning it to reduce the cost, it gave vague or irrelevant answers. What we found was that it is possible to get good results from it, but you have to spend so much time and data to train it, and the cost of training is an issue.
For another task, a document scanner, we tried GPT, and we got poor results. We had better results with Google, and Mistral, but the Google (I mean the Document AI) was doing good for very specific tasks that it is designed for, and basicly there is not much training, so even for that task of document scaning, it couldn't fulfill our need. Mistral-medium was better, but the cost issue is still there, and in order to get good results for a specific problem, it's recommended that you create your own custom model and host it somewhere (Mistral has a new model, mistral-large, that claims that is the most powerful model of mistral, but I haven't tested it).

To summarize, it's possible to get good results, but it's not a small task. It requires a good deal of time and money and effort.
 

I scratched the surface of my abundant ignorance about the subject of AI and will now stop proving my ignorance. I asked an AI the following -

can an AI summerize multiple books to then create a new book?

Yes, an AI can summarize multiple books and then create a new book based on the summaries. This process is known as "book summarization" or "text summarization". Here's a high-level overview of how it works:

1. Book selection: The AI system selects a large number of books that are relevant to a specific topic or theme.
2. Book summarization: The AI system uses a text summarization algorithm to summarize each book into a concise summary, usually a few paragraphs long.
3. Summary analysis: The AI system analyzes the summaries to identify common themes, patterns, and connections between the books.
4. New book creation: The AI system uses the analyzed summaries to create a new book that combines the key ideas, concepts, and insights from the original books.

Some examples of text summarization algorithms that can be used for book summarization include:

1. Latent Semantic Analysis (LSA): LSA is a statistical method that analyzes the relationship between words and their contexts to identify the most relevant information.
2. Latent Dirichlet Allocation (LDA): LDA is a topic modeling algorithm that identifies the underlying topics in a corpus of text and summarizes the text around those topics.
3. TextRank: TextRank is a graph-based algorithm that uses a ranking system to identify the most important sentences and phrases in a text.
4. Deep Learning-based Methods: Deep learning-based methods, such as Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs), can be used to summarize text by identifying the most salient features and patterns.

Some examples of AI systems that can summarize multiple books and create a new book include:

1. Book summarization tools: Book summarization tools, such as SummarizeBot and BookSummary, use natural language processing (NLP) and machine learning algorithms to summarize books.
2. AI-powered writing assistants: AI-powered writing assistants, such as AI Writer and Content Blossom, use NLP and machine learning algorithms to summarize books and create new content.
3. Book creation platforms: Book creation platforms, such as BookCreator and Novelize, use AI-powered tools to help authors create new books by summarizing and analyzing existing books.

Some examples of new books that can be created using AI-book summarization include:

1. Fiction novels: AI-book summarization can be used to create new fiction novels by summarizing existing books and combining the key elements to create a new story.
2. Non-fiction books: AI-book summarization can be used to create new non-fiction books by summarizing existing books and combining the key concepts and ideas to create a new book.
3. Research papers: AI-book summarization can be used to create new research papers by summarizing existing papers and combining the key findings and insights to create a new paper.

Overall, AI-book summarization is a powerful tool that can help authors and researchers create new books by summarizing and analyzing existing books.

 Siavash, what you say confirms my intuition that it would be a tedious and complex/long enough task as to require money to really happen. 

It's funny because people tend to assume this would be realtively easy, or if not easy, at least very inexpensive, and yet from what I've seen even with people being paid to do stuff, is that having something even basically correct, let alone original, is really hard. We had a few professionals doing principal components analysis on relatively simple phenomenological reports of progress of insight, and the results that were come up with in the time that someone can volunteer for, seemed basically random, to me.

But, @dreamwalker, not wanting to discourage you, I may be wrong, and I have a spreadsheet with the DhO messages up to 2019 or 2020, if you share your email I can share this as a good start?

​​​​​​​edited 2x 

I've been thinking about how and why to try using AI on DhO's message board database. I think it's not a good idea because:

1. DhO is a volunteer organization and no volunteer will put in the time and effort required (see Siavash's last post)

2. Money - no funding to pay for the subscriptions and fees that are required of using high-powered AI.

3. Dubious output - AI just isn't reliable enough right now.

4. Required manual effort to get DhO data into a usable condition. This HAS to come before anything else and will be an enormous, time-consuming task.

​​​​​​​5. Liferay - need I say more?

AI is being oversold by companies that really have no business model, like OpenAI, or that are running scared of not pushing AI and being left behind - but behind what? There are multiple copyright infringement lawsuits against these companies by major content providers, artists, and news publications. AI company CEOs are out over their skis - Sam Altman actually wants a $7 trillion (yep, TRILLION) investment for OpenAI. And he's serious as far as I can tell. It seems that progress in improving AI power, effectiveness, and reliability has stalled for some time now. AI sucks up a shit Tom of energy, too.

AI is fun to play with and it can provide nice but incremental improvements in daily work. I've talked to a lot of companies looking to make AI work for them and they all report what Siavash has said. Expensive. Cost-benefit is too negative. Unreliable and a major time sink.

So, is the gain for DhO really there for us, or are we fooling ourselves, leaving common sense out of our calculations because we were swept up in the hype?

I agree with the point you made Chris, that it's a useful tool, BUT, if someone has the knowledge and expertise on how to use it and how to distinguish the good output from the crap.

I observe it in our team, the more knowledgable and experienced developers take advantage of it, and reduce some tasks from let's say 6 hours to 1-2 hours by letting it do the search and generate the scaffolding for codes, but less experienced developers just get confused with the irrelevant content it generates.

I have also tried to use it for generating code for specific problems. So, what I do is to create a context, then expand the context, and expect it to learn and remember that the context is evolving. So I ask it about a possible solution for a problem, then ask it let's code it, it responds well for a few steps, then intentionally or unintentionally, it seems that it forgets, or ignores the contexts, and goes back to giving generic answers, or irrelevant answers.
It could be that it's configured to be this way, who knows. I heard Sam Altman and Bill Gates once saying that they know what data they have given it, but they don't know how it comes up with the answers.
I maybe tried 20-30 times to try to code with it, but I just gave up, but it's great for generating simple templates. And earlier iteration were better for situations like this (coding specific problem), especially with GPT.

I like mistral much more than GPT for conversations and code work.

1. tru dat...lazy we be, just sit there we do.
2. pick it off the dharma tree? it grows free right?
3. Dho data was dubious way before AI
4. DATA:   Not to waste your time Chris but I used to clean database "information" and migrate etc for a living back when....i'm rusty AF but curious -

• What is usable condition for data mean?
• Can liferay be exported into a database format that can be loaded? or the tables exported in any way to then import back into a database? I'f I got primary keys in a comma seperated text file. I can fiddle it. SQL and report writing twas also my jam with a bit o scripting...

• What does a liferay export look like anyway?

I'm curious about Step 1 at a time....(without admitting anything or invocing higher powers)
Thanks,
​​​​​​​~D

DreamWalker,

Exporting data from the mysql database of Liferay is not an issue.
How to interpret that, can be an issue, at least with GPT, since I've seen it get into trouble, when you have, for instance a comma separated data, but one of the cells itself can have a comma separated data, and to make it understand the indexing of the data, and that each cell can have a single or multiple value can become annoying and time-consuming, but in DhO, as far as I know, the data structure is very simple, and there shouldn't be much problem in that regard.
​​​​​​​

Hi!

What size are we talking about?
Easy to export as well as import again? The whole database or just the forum part or per selected tables?

I have NO experience of feeding AI in any useable format that it can eat...
but I would love to fiddle with the Dho data and run some queries against it based off of time, username, postlength, postcount, etc that might give me an interesting way to create possible ranking scores for highlighting quality post identification vs ways to score low ranking. Using that to then create subtables/reports to feed an AI to identify possible keywords to further allow sort/categorize....hmmm....the possibilities.

This might beat the hell out of reading every single post since the beginning of time....which I actually did back when. 
Does my permission level allow exporting?
Thanks,
​​​​​​​~D

 Hi DreamWalker!

Well, exporting is much easier than importing, both technically, and also because any import has the risk of unwanted modification in the data, and since its user data, that could be troubling.
And it depends on what you mean by import. If you mean dump the whole data, do some modification on it with some code, then import the whole database again, that should be easy.

When I saif DhO has simple structure, I meant the posts and categories tables. Liferay has many other tables in its portal, though most of them are not used in DhO, and most of DhO content is the categories, topics, messages, and users/roles/permissions.

For feeding data, one way is to feed the data and then send prompts to the AI and get responses. Another way is to introduce the data structure and data format, then ask the AI to generate queries, instead of writing manual queries to extract a subset of the data. Depending on how much time and money you want to spend, of course you can combine them. Use AI to generate queries, and refine queries, then include that in your instruction, and feed the data into it, and do it for a while to train it, but personally if I was a contractor, I can not put a number in front of the hours of work estimation .

 

I forgot to mention, by export I meant directly from the database, not from the Liferay portal.

No, it's not possible to export the data from the Liferay portal.

For me the question is not "how" to do a DhO AI, but why? What is the benefit beyond being able to query and read the actual conversations? AI typically waters the humanity and individual voices out of textual content. It's a sort of regression to the mean. I'm not sure that's helpful to anyone.

So, why AI? What's better than the original content by the human beings who contributed?

About the ranking, at least two kinds of ranking are needed, one for the quality of the data that is feeded, one for the responses that the AI generates. I don't remember about GPT, but some others, like Amazon and Google for the document AI, they give you a ranking about the result, but how can I trust the AI to evaluate its work!?
For ranking the data itself, any ranking logic you choose, it has complexities. You may have a poster that shows up once in the forum and posts a comment which is quite important. Sometimes such a comment start a discussion, and a series of topics, sometimes such a comment ends a discussion, and other times it doesn't have those effect. It's very difficult to extract such a data manually, and we would expect the AI to do a better job there.
​​​​​​​The ranking itself is one of the key problems to solve.

Well, it saves us the time to read the content .



( Oh, if someone doesn't have the time to read the content, are they serious about it?! )

Yes, Siavash. That's my point about the extreme difficulty of making DhO data useful in this context. It's a mass of complexity and opportunities to reduce the nuance present here to, well, irrelevancy.

EDIT: Here's another thought - if I can get an AI to provide all the practice-oriented information contained in the DhO message boards, why would I want to converse with other people here? It's possible that a DhO AI could end the DhO as it has existed for almost 20 years.

Further, knowing how asking AI for information is nuanced and complicated, and you can get the answer you want by telegraphing that in your query, what if the DhO AI turns out to be unreliable and, due to the potential lack of human interaction on the message boards, unreliable?

 I Asked mistral-medium :

Me:
Hi. I had a question. Are you familiar with DhO forum?

Mistral:
Yes, I'm familiar with the Dharma Overground (DhO) forum. It's an online community dedicated to discussing, practicing, and supporting the development of insight meditation, also known as vipassana or mindfulness meditation. The forum was founded by Daniel Ingram, a meditation teacher and author, who is known for his book "Mastering the Core Teachings of the Buddha" and his interpretation of the progress of insight meditation as a series of distinct stages. The DhO forum is a place where people can share their experiences, ask questions, and seek guidance from more experienced practitioners. It's important to note that while the forum can be a valuable resource for those interested in insight meditation, it's always a good idea to approach online communities with a critical and discerning mind, and to seek guidance from qualified teachers when possible.

Me:
There is a discussion there at the moment, about creating a DhO AI/LLM. Some of think that its against the idea of DhO, but others think that it would be great source of advice. What do you think? Considering the fact that DhO is run by volunteers, and doesn't have much financial resources.

Mistral:
Creating an AI or large language model (LLM) for the Dharma Overground (DhO) forum could have both benefits and drawbacks. On the one hand, an AI/LLM could potentially help to answer common questions, provide guidance, and support the community in a scalable and efficient way. It could also help to reduce the workload of volunteers and provide more consistent and accurate information. On the other hand, there are also potential risks and concerns associated with using AI/LLMs in a spiritual context. For example, there is a risk that the AI/LLM may not fully understand the nuances and complexities of meditation and spiritual practice, and may provide inaccurate or misleading advice. Additionally, there is a risk that the use of AI/LLMs may undermine the value of human connection, community, and personal guidance that is often central to spiritual practice. Given the limited financial resources of the DhO forum, it may be challenging to develop and maintain a high-quality AI/LLM that is able to meet the needs of the community. It may be more feasible to focus on building and strengthening the human community of volunteers, teachers, and practitioners, and to explore other ways to provide support and guidance that are more in line with the values and goals of the DhO forum. Ultimately, the decision about whether or not to develop an AI/LLM for the DhO forum should be made in consultation with the community and with careful consideration of the potential benefits and risks.
 

Example of contextual unreliability of LLMs:

Kenneth Folk believes Anthropic's LLM Claude is sentient and conscious. As he interacts with Claude he does so in a manner that elicits responses that cause the LLM to make itself claim sentience and consciousness. Yet, if I ask Claude directly, "Are you sentient and conscious?" it readily admits it is neither.

How might that play out in the context of the dharma?

I don't think an LLM/AI can be an experienced practitioner

Sure, it doesn't have the experience, or does it!?
Does it experience pain?
If I want someone's opinion on how to deal with the pain of the loss of a friend, I want it to be someone that has lost a friend and lived through that experience!

All its experiences are vicarious. They come from training data. It can probably mimic having had that experience.

I'm going to go to bed and dream of highly intelligent LLMs with agency and intent.

Well, after banging my head against the portal for 2 hours, I agree that I indeed failed to export successfully. Knowing what can't be done is useful...lol
​​​​​​​ps is the zip small enough to email?

I'm running down the idea of a better searchbot with AI helping summarize and helping beginners to ask better questions. Adding multiple rankings columns then populating each reasonably then given multiplyers, add it all up for sort list order for AI to use to summarize then list results by order....

Ranking would need reverse eng testing, doesn't preclude outliers just orders better.... Unlike now where gems are currently lost already.

Chris, could you do me a favour and ask the AI to meditate for 45-60 minutes and use the Noting Technuque of 1-10 notes per second (maybe faster even). It's to note matter of fact phenomena of course. Then report back about its experience in detail. 

Then ask it to meditate like this every day for 6 months and keep logging about it. 

I have a hunch all this is but an echo of our own voice and we expect too much from it. Does an echo have intelligence? 
Are we also nothing else but an echo? An echo of nothingness. 

A DJ sampling previous echos and making our own tunes.

Karmic Boogie-woogie! 

Hey, thanks for all of your thoughts.Steffan Iverson is the primary person who is wanting to help with this, and he already built an AI model based on my forum posts and MCTB2 found here: https://danielgpt.web.app  I will direct him to this thread to get a sense of the community vibe.Also, Claude is complex, and if you prompt with posts that imply Claude is sentient, it replies as if it is, and, if you do the opposite, or don't prompt in that direction, it replies as if it isn't, the ontological and existential implications of which can be interpreted various ways.Ok, I will facilitate the building of the DhO AI, see what it does, and people can comment on its usefulness, downsides, and we will make sure to add all appropriate warnings, along the lines of "This DhO AI is a totally experimental tool based on the Dharma Overground, which is full of contractions, differing views, schools of thought, a vast range of levels of practice, and has all the problems, perils, and promises of a public online forum. Proceed with caution, keep your wits about you, and don't take any of this too seriously."Thanks!Daniel 

Daniel, after reading all the caveats you just posted, why do this at all? What's the goal? I'm really and truly curious.

Also, the link you posted doesn't work. Is there an alternate?

https://danielgpt.web.app

Thanks, Jim. I'll fix Daniel's post, too. There were Word formatting hidden text characters in his link.

Just FYI, 

https://danielgpt.web.app

This is cool!

hanks to you and Steffan for making it!

1. Explain? what is 'all'?
2. Is there another way to experience an echo? Perhaps that is not our voice?
3. How? What is too much?
4. Do we have 'Intelligence'?
5. Dunno for sure, what is your theory?
6. Ummm, how did you leap to nothingness?

Stuff happens, on its own, where it is, but that don't mean it's nothingness...for whatever perspective that is able to perceive stuff.

We are all biological computers that reflect echos of our past experience/learning.
So what?
~D

This is kinda fascinating. How long until this allows direct inputs to AI training?....hmmm...MIND READING!!! Infancy, but an amazing start. 

Semantic reconstruction of continuous language from non-invasive brain recordings