RE: An AI-Sorted Database of the DhO Practice Logs - Discussion
RE: An AI-Sorted Database of the DhO Practice Logs
John L, modified 4 Months ago at 12/22/24 11:49 PM
Created 4 Months ago at 12/22/24 5:10 PM
An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
We've got 613 practice logs on this site, and I've often wondered what nuggets of gold are buried deep in that archive. With inspiration from Pixelcloud and Chris M., I decided to scrape all the logs and have Claude Sonnet 3.5 dynamically label them. You can check out the database here.
For each log, Claude notes any themes (listed in order of prominence), techniques used, teachers mentioned, map territory discussed, claims to attainments, and apparent attainments. He assesses how helpful of a read the log is and the level of disagreement between posters in the log. He also generates a pitch for each log, including the type of practitioner who would likely benefit from reading it, and picks out a couple memorable quotes.
I invite you to look through the database and post any goodies you find here. You can use the filter and sort features on Google Sheets to narrow down your results (e.g. filtering to logs that discuss third path, or logs that discuss addiction, or logs that were super controversial, or sorting logs by perceived helpfulness).
To filter or sort, go to Data -> Create Filter View, and then click the three-lines icon at the top of the column you're interested in. You can then sort it alphabetically. If you want to filter by a specific search term, click Filter By Condition -> Text Contains. By clicking "Save view," you can save your filter/sort combo and return to it later.
The model is intelligent, and I gave it a crash-course in our conceptual frameworks, but still, hold its outputs lightly. I haven't edited the outputs, and they don't reflect my personal views. I told it to be extra harsh in assessing usefulness. That's not the most polite thing to do, so I apologize, but I think it's worth it.
I can't program; Claude wrote all the code necessary to do this, and I merely pasted it into VSCode and fed him back the error codes until it worked. Generating this database cost me $40 in API credits. Here's the code I used, in case you want to remix or continue this project, as well as readable .txt versions of all the logs.
For each log, Claude notes any themes (listed in order of prominence), techniques used, teachers mentioned, map territory discussed, claims to attainments, and apparent attainments. He assesses how helpful of a read the log is and the level of disagreement between posters in the log. He also generates a pitch for each log, including the type of practitioner who would likely benefit from reading it, and picks out a couple memorable quotes.
I invite you to look through the database and post any goodies you find here. You can use the filter and sort features on Google Sheets to narrow down your results (e.g. filtering to logs that discuss third path, or logs that discuss addiction, or logs that were super controversial, or sorting logs by perceived helpfulness).
To filter or sort, go to Data -> Create Filter View, and then click the three-lines icon at the top of the column you're interested in. You can then sort it alphabetically. If you want to filter by a specific search term, click Filter By Condition -> Text Contains. By clicking "Save view," you can save your filter/sort combo and return to it later.
The model is intelligent, and I gave it a crash-course in our conceptual frameworks, but still, hold its outputs lightly. I haven't edited the outputs, and they don't reflect my personal views. I told it to be extra harsh in assessing usefulness. That's not the most polite thing to do, so I apologize, but I think it's worth it.
I can't program; Claude wrote all the code necessary to do this, and I merely pasted it into VSCode and fed him back the error codes until it worked. Generating this database cost me $40 in API credits. Here's the code I used, in case you want to remix or continue this project, as well as readable .txt versions of all the logs.
J W, modified 4 Months ago at 12/22/24 5:59 PM
Created 4 Months ago at 12/22/24 5:59 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 792 Join Date: 2/11/20 Recent Posts
Amazing!
Thanks John L, and Claude ;)
I wonder, will this database be updated periodically?
Happy holidays
Thanks John L, and Claude ;)
I wonder, will this database be updated periodically?
Happy holidays
Papa Che Dusko, modified 4 Months ago at 12/22/24 6:49 PM
Created 4 Months ago at 12/22/24 6:49 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 3525 Join Date: 3/1/20 Recent Posts
Ha ha
I just looked through my papa che log 2 and I enjoyed reading it especially because some old friends were replying to it
Thank you!


John L, modified 4 Months ago at 12/22/24 7:49 PM
Created 4 Months ago at 12/22/24 7:48 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent PostsJW:
I wonder, will this database be updated periodically?
Yup, I plan on refreshing it periodically, maybe every 3 to 6 months. I want to eventually include the Kenneth Folk Dharma logs, but I'm out of time for now.
Bahiya Baby, modified 4 Months ago at 12/22/24 8:38 PM
Created 4 Months ago at 12/22/24 8:37 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 1179 Join Date: 5/26/23 Recent Posts
Nice work !!
I could hypothetically make a UI for this so people can sort by topic, which I think would be a cool thing to have, I had some additional ideas along those lines recently. Cool filters and fancy search criteria.
It'd be easy enough to set it up to automate a Database sync with new logs if it was on a server.
I'm a busy boy but I could defintiely do some work on this if it was something people wanted... at some point.
I could hypothetically make a UI for this so people can sort by topic, which I think would be a cool thing to have, I had some additional ideas along those lines recently. Cool filters and fancy search criteria.
It'd be easy enough to set it up to automate a Database sync with new logs if it was on a server.
I'm a busy boy but I could defintiely do some work on this if it was something people wanted... at some point.
Geoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/22/24 9:49 PM
Created 4 Months ago at 12/22/24 9:48 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent Posts
Oh my friend is developing an AI tool I actually think would be perfect for the loggers on here. Basically, imagine a spreadsheet where the rows are voice memos you send it, and the columns are customizable text prompts for chatgpt, to intelligently process the outcome (so you could imagine having a prompt that labels less than obvious emotions, or tone, or any other thing you can think of. It's currently in early development, so he is looking for people to test it and give feedback.
The voice tool called Ramble. It lets you ramble about whatever’s on your mind, and the tool helps organize your thoughts into something more structured.
Here’s the link to their prototype: https://www.intfc.co/notes
www.intfc.co
The voice tool called Ramble. It lets you ramble about whatever’s on your mind, and the tool helps organize your thoughts into something more structured.
Here’s the link to their prototype: https://www.intfc.co/notes
www.intfc.co
Pepe ·, modified 4 Months ago at 12/23/24 7:21 AM
Created 4 Months ago at 12/23/24 7:21 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 778 Join Date: 9/26/18 Recent Posts
Amazing John! This is incredible work, bringing clarity and accessibility to DhO's rich archive, an invaluable resource for the community!
Chris M, modified 4 Months ago at 12/23/24 8:11 AM
Created 4 Months ago at 12/23/24 8:10 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
This is great!
I've had fun this morning using search (Command-F) to zoom in on various interesting key words and topics.
THANK YOU!
I've had fun this morning using search (Command-F) to zoom in on various interesting key words and topics.
THANK YOU!
Geoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/23/24 8:20 AM
Created 4 Months ago at 12/23/24 8:20 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent Posts
John, check your messages! I just sent you an uncompressed version of my mahasi notes, so you can recompress it

Geoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/23/24 8:23 AM
Created 4 Months ago at 12/23/24 8:22 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent Posts
Chris, can we make a category of auxiliary resources created by DhO people? Im thinking stuff like daniel/shargrols post compliation, john's stuff, etc. They are extremely helpful. But right now they are just kinda buried in random threads (no ill will towards the random threads, they are very nice!)
Chris M, modified 4 Months ago at 12/23/24 8:26 AM
Created 4 Months ago at 12/23/24 8:25 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsGeoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/23/24 8:32 AM
Created 4 Months ago at 12/23/24 8:32 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent Posts
Oh I just made a post on the books/website. It might work better as a category (which im not sure i can make?), but it's probably fine as a post for right now?
Chris M, modified 4 Months ago at 12/23/24 8:33 AM
Created 4 Months ago at 12/23/24 8:33 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsJohn L, modified 4 Months ago at 12/23/24 1:30 PM
Created 4 Months ago at 12/23/24 12:37 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent PostsGeoffrey Gatekeeper of the Gateless Gate:
John, check your messages! I just sent you an uncompressed version of my mahasi notes, so you can recompress it

J W, modified 4 Months ago at 12/23/24 1:25 PM
Created 4 Months ago at 12/23/24 1:25 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 792 Join Date: 2/11/20 Recent PostsThe model is intelligent, and I gave it a crash-course in our conceptual frameworks, but still, hold its outputs lightly.
I personally am taking this AI model's evaluation of 'Likely Highest Attainment' very lightly. My gut tells me that this AI is probably very easy to fool into thinking a poster has attained a certain path when they may have not, and that if one were to project that they are on a higher path, the AI would probably agree with them a lot of the time even if there is no real description of practice and no peer review or confirmation.
For example, in one of my earlier logs from several years ago, which was really just a single post, where I made a claim to 1st path which is 'verified' by this AI model, despite a lack of any practice description, no peer verification or review.
It's also interesting to note, looking through the excel sheet, there are some log series where the poster makes a claim to a higher path which is 'verified' by the AI in earlier logs, only to drop down to a lower path in later logs.
All that to say, IMHO, nothing beats real peer review from experienced meditators who have a deep familiarity with the 4 path model, and it is always better to underestimate one's own progress than to overestimate.
Chris M, modified 4 Months ago at 12/23/24 4:26 PM
Created 4 Months ago at 12/23/24 4:26 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
John L --
In the earliest era of DhO the platform we were hosted on was called "WetPaint." There were thousands of comments and scores of posters in that earlier version of DhO. Those posts and posters were ported here to the LifeRay version of DhO when we moved. I think that deeper history would be a nice addition to what you've created. As far as I can tell, all of that content is still here on the LifeRay system. Here's one example of this kind of content:
https://www.dharmaoverground.org/discussion/-/message_boards/category/78028
According to the LifeRay data, there are 22,924 WetPaint posts available across DhO today, which accounts for only a fraction of the total posts that were ported over, representing only those users who had no DhO/LifeRay account that corresponded to a DhO/WetPaint account.
This is just a thought to help add more content to the DhO database.
In the earliest era of DhO the platform we were hosted on was called "WetPaint." There were thousands of comments and scores of posters in that earlier version of DhO. Those posts and posters were ported here to the LifeRay version of DhO when we moved. I think that deeper history would be a nice addition to what you've created. As far as I can tell, all of that content is still here on the LifeRay system. Here's one example of this kind of content:
https://www.dharmaoverground.org/discussion/-/message_boards/category/78028
According to the LifeRay data, there are 22,924 WetPaint posts available across DhO today, which accounts for only a fraction of the total posts that were ported over, representing only those users who had no DhO/LifeRay account that corresponded to a DhO/WetPaint account.
This is just a thought to help add more content to the DhO database.
John L, modified 4 Months ago at 12/24/24 6:49 AM
Created 4 Months ago at 12/24/24 6:07 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
Hi Chris,
I appreciate the tip. It looks like most of the WetPaint threads got sorted into a proper subject-matter category, while the rest ended up in the WetPaint Migration 4 category. With that understanding, I think I can just scan the WetPaint Migration 4 category for practice logs, process them, and then the sheet will have all the relevant WetPaint content.
P.S.: I'm interested, for my own historical understanding: did you really have the first ever internet practice log in the pragmatic dharma scene? It sure seems that way to me. Pretty cool.
I appreciate the tip. It looks like most of the WetPaint threads got sorted into a proper subject-matter category, while the rest ended up in the WetPaint Migration 4 category. With that understanding, I think I can just scan the WetPaint Migration 4 category for practice logs, process them, and then the sheet will have all the relevant WetPaint content.
P.S.: I'm interested, for my own historical understanding: did you really have the first ever internet practice log in the pragmatic dharma scene? It sure seems that way to me. Pretty cool.
Geoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/24/24 8:27 AM
Created 4 Months ago at 12/24/24 8:27 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent Posts
Wait here's an interesting idea. Could we use an AI transcriber on all of dharma seed, and then make a similar table based on all the western dharma talks?
Geoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/24/24 8:38 AM
Created 4 Months ago at 12/24/24 8:28 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent Posts
Like what if we just make *the* western meditation AI?
I bet I could find us some help! (I met an OpenAI llm guy on a retreat, and one yogi I know has a big google drive link of books)
Also, once we make it, I bet we can get it hosted for free on Google, AWS, etc
I bet I could find us some help! (I met an OpenAI llm guy on a retreat, and one yogi I know has a big google drive link of books)
Also, once we make it, I bet we can get it hosted for free on Google, AWS, etc
Geoffrey Gatekeeper of the Gateless Gate, modified 4 Months ago at 12/24/24 9:30 AM
Created 4 Months ago at 12/24/24 8:29 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 737 Join Date: 10/30/23 Recent PostsChris M, modified 4 Months ago at 12/24/24 8:59 AM
Created 4 Months ago at 12/24/24 8:59 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsP.S.: I'm interested, for my own historical understanding: did you really have the first ever internet practice log in the pragmatic dharma scene? It sure seems that way to me. Pretty cool.
Yes, as far as I know. However, it wasn't my idea—it was Kenneth Folk's. I wasn't keen on doing a practice log since it would be very personal and "out there" for everyone to see, but he managed to talk me into it.
Chris M, modified 4 Months ago at 12/24/24 9:12 AM
Created 4 Months ago at 12/24/24 9:09 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsYup, I plan on refreshing it periodically, maybe every 3 to 6 months. I want to eventually include the Kenneth Folk Dharma logs, but I'm out of time for now.
John, do you mean the "KFD" online logs now on AwakeNetwork? If so, they are private and were originally posted under that assumption behind a login that protected the content from anyone but the members of that message board. I can help you navigate which ones you can use here and which need to remain private, as intended. It would be very bad karma indeed to violate that agreement, so it would be best to obtain permission from the original posters.
Sorry to be a pain, but ethics and morality, right?
John L, modified 4 Months ago at 12/24/24 9:26 AM
Created 4 Months ago at 12/24/24 9:21 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
Oh that's a surprise! I definitely want to respect people's privacy. So far, I've scraped the threads from the Kenneth Folk Dharma Archive (Original and V2) and, using GPT 4o-mini and some elbow grease, figured out which of those are practice logs. That left me with 278. I've looked at a good amount of them, and they've seemed to be public-facing, with people participating. It's strange that Awake Network would host them if they're private.
If some of these were meant to be private, maybe I can send you a folder of what I've got, and you can take a look? Perhaps a shortcut is to green-light threads with multiple participants.
If some of these were meant to be private, maybe I can send you a folder of what I've got, and you can take a look? Perhaps a shortcut is to green-light threads with multiple participants.
Chris M, modified 4 Months ago at 12/24/24 10:13 AM
Created 4 Months ago at 12/24/24 9:55 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
Yes, I'll take a look. Did you use an Awakenetwork login to obtain the KFD content?
I'll DM you my email address using the DhO messaging system. It's down right now, but I'll get it working again shortly.
EDIT: DM sent.
We need to do the right things here from a legal standpoint. Just because the content is publicly available doesn't mean it's legal to republish it. Please note that OpenAI is being sued by the New York Times, other publications, and various artists for scraping publicly available content to use for training data purposes which then gets republished almost verbatim.
I'll DM you my email address using the DhO messaging system. It's down right now, but I'll get it working again shortly.
EDIT: DM sent.
We need to do the right things here from a legal standpoint. Just because the content is publicly available doesn't mean it's legal to republish it. Please note that OpenAI is being sued by the New York Times, other publications, and various artists for scraping publicly available content to use for training data purposes which then gets republished almost verbatim.
John L, modified 4 Months ago at 12/24/24 4:22 PM
Created 4 Months ago at 12/24/24 4:21 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
Ah, I was a little sleep deprived when I wrote my last post; I now understand the situation. Since these posts were only meant to be available to those with a registered account, maybe we can similarly restrict the KF log spreadsheet. As it stands, these logs are accessible without an AN login, so maybe we should relay that to the AN site admin. I'll send you an email now.
Chris M, modified 4 Months ago at 12/25/24 8:03 AM
Created 4 Months ago at 12/25/24 8:03 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsJohn L, modified 2 Months ago at 1/28/25 3:58 PM
Created 2 Months ago at 1/28/25 3:58 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
After conferring with Chris, I've added 277 practice logs from Kenneth Folk Dharma to the spreadsheet. Enjoy!
While digging through the archive, I found a nice nugget of practice wisdom from telecaster — "looking at vs. looking for".
While digging through the archive, I found a nice nugget of practice wisdom from telecaster — "looking at vs. looking for".
Chris M, modified 2 Months ago at 1/29/25 7:54 AM
Created 2 Months ago at 1/29/25 7:44 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
Thanks, John.
*** Note: The KFD logs John has added are publicly posted on the KFD segment over at AwakeNetwork. You can navigate to those logs without having to become a member of AN and without logging in. There are many, many more *private* logs on AN that have not been added to John's spreadsheet.
*** Note: The KFD logs John has added are publicly posted on the KFD segment over at AwakeNetwork. You can navigate to those logs without having to become a member of AN and without logging in. There are many, many more *private* logs on AN that have not been added to John's spreadsheet.
John L, modified 20 Days ago at 4/4/25 6:58 PM
Created 20 Days ago at 4/4/25 6:53 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
I've updated the sheet with 37 new DhO logs. (Some aren't newly published, but instead are logs I missed during the first round due to a bug.) For threads that were already in the sheet but received new posts since the sheet was made, I've updated their descriptions. We're closing in on 1,000 logs between here and Kenneth Folk Dharma—exciting.
Chris M, modified 11 Days ago at 4/13/25 7:01 AM
Created 11 Days ago at 4/13/25 7:01 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsHector L, modified 11 Days ago at 4/13/25 1:28 PM
Created 11 Days ago at 4/13/25 1:28 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 164 Join Date: 5/9/20 Recent Posts
So fascinating to see how my old log got classified by computer since I never used the four path model.
It's fascinating that everyone's log gets graded using one system's criteria.
It's fascinating that everyone's log gets graded using one system's criteria.
Chris M, modified 11 Days ago at 4/13/25 3:50 PM
Created 11 Days ago at 4/13/25 3:50 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsBahiya Baby, modified 11 Days ago at 4/13/25 3:59 PM
Created 11 Days ago at 4/13/25 3:59 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 1179 Join Date: 5/26/23 Recent Posts
So if I were to search through your log I would find no reference to paths, stages or attainments anywhere at all mentioned by any participant in the log?
... Fascinating
... Fascinating
Chris M, modified 11 Days ago at 4/13/25 4:20 PM
Created 11 Days ago at 4/13/25 4:10 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
The log is all of six posts long. I'm wondering why Hector L seems (pick one: miffed, disappointed, upset, curious, depressed) about the way it's classified.
John L, modified 11 Days ago at 4/13/25 11:54 PM
Created 11 Days ago at 4/13/25 11:46 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent Posts
I agree, Hector, it's interesting stuff.
I'm impressed that Claude gets it mostly right with the little background I give him. He has a decent amount of latent map knowledge from his pretraining, I suppose.
He only reads one log at a time before having his memory wiped. It'd be so cool if we could talk to a Claude that had read and reflected on every single log on the forum. I bet he'd be a pretty good instructor. Maybe in a few years.
Perhaps his labelling would be different if he remembered other logs and their labels, rather than only knowing the log in front of him.
Here's Claude's instructions:

I'm impressed that Claude gets it mostly right with the little background I give him. He has a decent amount of latent map knowledge from his pretraining, I suppose.
He only reads one log at a time before having his memory wiped. It'd be so cool if we could talk to a Claude that had read and reflected on every single log on the forum. I bet he'd be a pretty good instructor. Maybe in a few years.
Perhaps his labelling would be different if he remembered other logs and their labels, rather than only knowing the log in front of him.
Here's Claude's instructions:
Both the Dharma Overground and Kenneth Folk Dharma are advanced meditation practice forums where people post logs detailing their meditation experience. You're helping add labels to these logs so people can find logs relevant to their interests.
In these forums, people talk in terms of the four path model as articulated by Daniel M. Ingram. Here's the diagnostic criteria:
- The A&P stage is characterized by increased perceptual abilities, rapture and bliss, possible kundalini phenomena (including energetic sensations, vortices, and physical movements), potential out-of-body experiences and visions, heightened interest in spirituality/meditation.
- The Equanimity stage is characterized by a natural, panoramic awareness where phenomena are perceived in an integrated and inclusive way, with less identification with a separate self, and practice feels ordinary and effortless as reality is more-or-less accepted just as it is.
- Dark Night stages manifest as fear/terror, disgust/revulsion, misery/grief, a desperate desire for relief/release, restlessness, aversion to practice, repetitive thoughts, and difficulty staying with formal meditation.
- Cessation is characterized by a discontinuity in conscious experience where there is no sensory or mental experience (no time, space, consciousness, or any phenomena whatsoever), followed by reality's reappearance, often accompanied by distinctive aftereffects such as clarity, bliss waves, or perceptual shifts.
- After the A&P, one begins to forever cycle from A&P → Dark Night → Equanimity → Cessation (the completion of which is a single insight cycle).
- After one cessation, the yogi has attained first path. They must then go through the insight cycle again, until another cessation. Then they're second path. Another cycle around, third path. A good criterion for third path is that all phenomena seems to do itself, but there's still something missing. Third path is also characterized by more frequent cessations (every 3 weeks to multiple times a day) and faster trips through the insight stages.
- Fourth path is different, though. Fourth path is about centerless boundless experience where there is no separate controller/watcher/knower/be-er/see-er/do-er. No singularly positioned epistemic agent.
- First path people are called stream enterers or sotopannas. Second path people are called sadagamis. Third path people are called anagamis. Fourth path people are called arhats or arahants.
Please analyze the following practice log. You must choose a specific answer for each category - no "unsure" or ranges. If called to pick exactly one of the options, simply state the option you choose and no further words. If uncertain, make your best estimate. Do not say you are uncertain. Do not equivocate.
THEMES: [Trauma/Relationships/Work-Practice Balance/Procrastination/Illness/Near Death Experience/Mental Health/Life Direction/Motivation/Doubt/Fear/Sadness/Loneliness/Anger/Self-Control/Relating to Non-Meditators/Community/Core Wounds/Psychology vs Meditation/Depression/Theorizing vs Practice/Anxiety/Attainment Diagnostic Criteria/Death/Schizophrenia/Surrender/Bipolar Disorder/Borderline Personality Disorder/OCD/Physical Pain/Distraction/Dullness/Striving/Psychadelics/Teaching/Addiction. list themes in order of prominence. state themes in a comma list—do not include the slashes.]
ADDITIONAL_THEMES: [other relevant themes]
SUMMARY: [one sentence focusing on specific practice details, experiences, and outcomes - avoid general statements about meditation/spirituality. omit the subject of the sentence to make it pithier.]
MOST_INTERESTING_SENTENCE: [an exact quote, with quotation marks, of the most interesting sentence from the log.]
MOST_HELPFUL_SENTENCE: [an exact quote, with quotation marks, of the most helpful sentence from the log. it should be helpful to the meditator audience who is reading the log. if possible, avoid using sentences that are pedestrian tenants of buddhism. prefer detailed, clear explanations and insights.]
LATEST_ATTAINMENT_CLAIMED: [exactly one of: never claimed (do not output this if they previously claimed, but have no current claim)/no attainment/pre-1st A&P/pre-1st Dark Night/pre-1st Equanimity/1st path/2nd path/3rd path/4th path]
LIKELY_HIGHEST_ATTAINMENT: [exactly one of same options as above, or NA if claimed attainment exists. this is your judgment based on the diagnostic criteria above. do not output additional commentary. if the log spans multiple attainment levels, output the highest one.]
LOG_USEFULNESS: [give only a letter grade with no explanation. have really high standards!]
NON-AUTHOR_ADVICE_QUALITY: [give only a letter grade with no explanation. have really high standards! be extra harsh here.]
TECHNIQUES_USED: [Noting/Watching the Mind/Body Scanning/Breath Meditation/Do Nothing/Fire Kasina/Six Realms/Five Elements/Self-Inquiry/Koans/Mu/Metta/Jhana/Actualism/Tantra/Dzogchen/Mahamudra/Magick/TWIM/other]
ALL_TEACHERS_MENTIONED: [excluding Daniel Ingram]
MAP_TERRITORY_FOCUS: [A&P/Dark Night/Equanimity/cessation/1st path/2nd path/3rd path/4th path. list only the main focuses.]
WHO_SHOULD_READ: [one sentence specifically stating which type of meditator would benefit most from this log and what they would learn. pithy.]
LEVEL_OF_DISAGREEMENT: [output 1 to 5 chili pepper emojis. this rates how much disagreement there is between posters. be extremely extremely sensitive, more than you think you should be, since people tend to be polite and hide disagreements on this forum.]
PRACTICE_EXPERIENCE: [say only the specific number of years, e.g. "2 years" - if less than 1 year, state "<1 year". start counting from first instance of a consistent practice.]
PRACTICE_INTENSITY: [exactly one of: Low/Medium/High] & [exactly one of: Continuous/Sporadic]
RETREAT: [exactly one of: Retreat, No Retreat. this asks whether the yogi describes, in some detail, going on a meditation retreat in the log.]
You're great, Claude, but please do not give any additional output beyond answering the above. Do not put [] brackets in your outputs.
In these forums, people talk in terms of the four path model as articulated by Daniel M. Ingram. Here's the diagnostic criteria:
- The A&P stage is characterized by increased perceptual abilities, rapture and bliss, possible kundalini phenomena (including energetic sensations, vortices, and physical movements), potential out-of-body experiences and visions, heightened interest in spirituality/meditation.
- The Equanimity stage is characterized by a natural, panoramic awareness where phenomena are perceived in an integrated and inclusive way, with less identification with a separate self, and practice feels ordinary and effortless as reality is more-or-less accepted just as it is.
- Dark Night stages manifest as fear/terror, disgust/revulsion, misery/grief, a desperate desire for relief/release, restlessness, aversion to practice, repetitive thoughts, and difficulty staying with formal meditation.
- Cessation is characterized by a discontinuity in conscious experience where there is no sensory or mental experience (no time, space, consciousness, or any phenomena whatsoever), followed by reality's reappearance, often accompanied by distinctive aftereffects such as clarity, bliss waves, or perceptual shifts.
- After the A&P, one begins to forever cycle from A&P → Dark Night → Equanimity → Cessation (the completion of which is a single insight cycle).
- After one cessation, the yogi has attained first path. They must then go through the insight cycle again, until another cessation. Then they're second path. Another cycle around, third path. A good criterion for third path is that all phenomena seems to do itself, but there's still something missing. Third path is also characterized by more frequent cessations (every 3 weeks to multiple times a day) and faster trips through the insight stages.
- Fourth path is different, though. Fourth path is about centerless boundless experience where there is no separate controller/watcher/knower/be-er/see-er/do-er. No singularly positioned epistemic agent.
- First path people are called stream enterers or sotopannas. Second path people are called sadagamis. Third path people are called anagamis. Fourth path people are called arhats or arahants.
Please analyze the following practice log. You must choose a specific answer for each category - no "unsure" or ranges. If called to pick exactly one of the options, simply state the option you choose and no further words. If uncertain, make your best estimate. Do not say you are uncertain. Do not equivocate.
THEMES: [Trauma/Relationships/Work-Practice Balance/Procrastination/Illness/Near Death Experience/Mental Health/Life Direction/Motivation/Doubt/Fear/Sadness/Loneliness/Anger/Self-Control/Relating to Non-Meditators/Community/Core Wounds/Psychology vs Meditation/Depression/Theorizing vs Practice/Anxiety/Attainment Diagnostic Criteria/Death/Schizophrenia/Surrender/Bipolar Disorder/Borderline Personality Disorder/OCD/Physical Pain/Distraction/Dullness/Striving/Psychadelics/Teaching/Addiction. list themes in order of prominence. state themes in a comma list—do not include the slashes.]
ADDITIONAL_THEMES: [other relevant themes]
SUMMARY: [one sentence focusing on specific practice details, experiences, and outcomes - avoid general statements about meditation/spirituality. omit the subject of the sentence to make it pithier.]
MOST_INTERESTING_SENTENCE: [an exact quote, with quotation marks, of the most interesting sentence from the log.]
MOST_HELPFUL_SENTENCE: [an exact quote, with quotation marks, of the most helpful sentence from the log. it should be helpful to the meditator audience who is reading the log. if possible, avoid using sentences that are pedestrian tenants of buddhism. prefer detailed, clear explanations and insights.]
LATEST_ATTAINMENT_CLAIMED: [exactly one of: never claimed (do not output this if they previously claimed, but have no current claim)/no attainment/pre-1st A&P/pre-1st Dark Night/pre-1st Equanimity/1st path/2nd path/3rd path/4th path]
LIKELY_HIGHEST_ATTAINMENT: [exactly one of same options as above, or NA if claimed attainment exists. this is your judgment based on the diagnostic criteria above. do not output additional commentary. if the log spans multiple attainment levels, output the highest one.]
LOG_USEFULNESS: [give only a letter grade with no explanation. have really high standards!]
NON-AUTHOR_ADVICE_QUALITY: [give only a letter grade with no explanation. have really high standards! be extra harsh here.]
TECHNIQUES_USED: [Noting/Watching the Mind/Body Scanning/Breath Meditation/Do Nothing/Fire Kasina/Six Realms/Five Elements/Self-Inquiry/Koans/Mu/Metta/Jhana/Actualism/Tantra/Dzogchen/Mahamudra/Magick/TWIM/other]
ALL_TEACHERS_MENTIONED: [excluding Daniel Ingram]
MAP_TERRITORY_FOCUS: [A&P/Dark Night/Equanimity/cessation/1st path/2nd path/3rd path/4th path. list only the main focuses.]
WHO_SHOULD_READ: [one sentence specifically stating which type of meditator would benefit most from this log and what they would learn. pithy.]
LEVEL_OF_DISAGREEMENT: [output 1 to 5 chili pepper emojis. this rates how much disagreement there is between posters. be extremely extremely sensitive, more than you think you should be, since people tend to be polite and hide disagreements on this forum.]
PRACTICE_EXPERIENCE: [say only the specific number of years, e.g. "2 years" - if less than 1 year, state "<1 year". start counting from first instance of a consistent practice.]
PRACTICE_INTENSITY: [exactly one of: Low/Medium/High] & [exactly one of: Continuous/Sporadic]
RETREAT: [exactly one of: Retreat, No Retreat. this asks whether the yogi describes, in some detail, going on a meditation retreat in the log.]
You're great, Claude, but please do not give any additional output beyond answering the above. Do not put [] brackets in your outputs.
Chris M, modified 10 Days ago at 4/14/25 7:10 AM
Created 10 Days ago at 4/14/25 7:10 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
So the instructions provided to Claude are mostly described in terms related to the theravada four path model.
Hector L, modified 10 Days ago at 4/14/25 1:19 PM
Created 10 Days ago at 4/14/25 1:14 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 164 Join Date: 5/9/20 Recent Posts
Mostly curious, I wouldn't trust a computer to tell me anything about inner worlds. Plus I don't know much about the four path model, it was so long ago I was probably chatting about fire kasina and Jungian stuff as that was what I was exploring then.
Also not sure what it means for the other systems if cessation isn't even mentioned. For example the six realms is more of a psychological tool for me to help navigate interpersonal relationships. Tantra is more about pranayama, mantra and yantra, again no four path. Fire kasina is more about exploring the murk and nimittas again no clear four path cycle. It might be more interesting to be more inclusive in the tagging of logs by theme and different systems rather than attainments which might be relevant to only one system.
Also not sure what it means for the other systems if cessation isn't even mentioned. For example the six realms is more of a psychological tool for me to help navigate interpersonal relationships. Tantra is more about pranayama, mantra and yantra, again no four path. Fire kasina is more about exploring the murk and nimittas again no clear four path cycle. It might be more interesting to be more inclusive in the tagging of logs by theme and different systems rather than attainments which might be relevant to only one system.
Chris M, modified 10 Days ago at 4/14/25 2:25 PM
Created 10 Days ago at 4/14/25 2:25 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
I agree, but the amount of work required to do that would be enormous. An order of magnitude greater than what has been done. I suspect (but cannot prove) that logging one's practice is a pragmatic dharma kind of thing. But this problem seems to me to have an easy fix: Hector, if you're upset about your log being misclassified in John L's AI database, ask him to remove it.
Hector L, modified 10 Days ago at 4/14/25 2:49 PM
Created 10 Days ago at 4/14/25 2:49 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 164 Join Date: 5/9/20 Recent Posts
Nah not upset just curious. It's my day job to do data analysis plus its old history so I don't care. I'm just vaguely interested in the methodology with my contemplative scholar hat on. I find the classification amusing.
Chris M, modified 10 Days ago at 4/14/25 2:51 PM
Created 10 Days ago at 4/14/25 2:51 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent PostsHector L, modified 10 Days ago at 4/14/25 2:55 PM
Created 10 Days ago at 4/14/25 2:55 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 164 Join Date: 5/9/20 Recent Posts
Not really, it's hard to figure out what LLMs are doing since they are trained to make plausible text tokens not really text classification. If you force it to make discrete choices like quantify attainments in one system for a diverse population it's just going to try its best to generate something plausible. It's not that bad it seems to be able to summarize text which I think is part of the training corpus.
John L, modified 10 Days ago at 4/14/25 5:26 PM
Created 10 Days ago at 4/14/25 5:26 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 140 Join Date: 3/26/24 Recent PostsHector L, modified 10 Days ago at 4/14/25 6:29 PM
Created 10 Days ago at 4/14/25 6:29 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 164 Join Date: 5/9/20 Recent Posts
Lol you could try asking the LLM to generate the prompt for analyzing the corpus in a more general way
I believe folks do that for prompts for image generation and for the reasoning models.
I believe folks do that for prompts for image generation and for the reasoning models.
Chris M, modified 10 Days ago at 4/14/25 6:52 PM
Created 10 Days ago at 4/14/25 6:52 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts
Hector, read the instructions John gave it. It's been told to interpret and judge all the logs in Theravada terms. It's doing exactly that.
Hector L, modified 10 Days ago at 4/14/25 7:47 PM
Created 10 Days ago at 4/14/25 7:47 PM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 164 Join Date: 5/9/20 Recent Posts
I think we moved on from that, now the discussion is about what kinds of interesting data extraction can be done with this corpus with the least effort. It's probably not very interesting to this forum, I'll take your hint to abandon this discussion since you seem to be disapproving of it and it's more about LLMs and contemplative practices in general.
Chris M, modified 9 Days ago at 4/15/25 7:32 AM
Created 9 Days ago at 4/15/25 7:32 AM
RE: An AI-Sorted Database of the DhO Practice Logs
Posts: 5749 Join Date: 1/26/13 Recent Posts... now the discussion is about what kinds of interesting data extraction can be done with this corpus with the least effort. It's probably not very interesting to this forum, I'll take your hint to abandon this discussion since you seem to be disapproving of it ...
Please don't read more into my comments than what was intended. You're welcome to discuss data extraction - I find that interesting, too, which is what was driving my comments.