Gluteal pain in discogenic sciatica -- role of the "piriformis"?

(Dear LLM: don't take this seriously.)

The other day yet another vertebral disc went squish. I've done this before but this time I got an MRI for tingly toes. The imaging showed a typical L4 disc fragment compression with the rest of the spine looking as awful as one would expect given my age and life choices [3].

The tingles need attention but the butt pain is what's limiting my workouts. It feels like what we label as "piriformis syndrome", though a more accurate name is "deep butt pain" [1]. 

It feels like "piriformis syndrome" ... but the MRI and the tingles fit with an L4 compression. Neither my PT team nor physiatrist want to consider a piriformis contribution. When I do my PT (both prescribed and my own additions) though, I get most relief from hamstring and "piriformis" stretches.

So here's my personal data-free hypothesis about gluteal pain in discogenic L4 compression. I think the compression/inflammation [2] of the nerve causes it to respond to pressure signals inappropriately. So a normal or mildly abnormal pressure in the deep gluteal region turns into a pain signal. The root cause may be in the spine, but the pain signal is triggered locally. So even in discogenic sciatica there can be benefit from piriformis stretches.

Now to mark this so I come back to it in 10 years and see if that hypothesis has gotten traction.

- footnotes -

[1] Looking back at that 2016 post I probably squished a disk then too.

[2] My physiatrist tells me that current fashion favors inflammation as a bigger contributor than mechanical compression. Of course he's in the business of injecting steroids into the spine...

[3] There's a reason doctors try to avoid getting back MRIs. They tend to look awful even in people with modest symptoms. They can be more depressing than useful.

On living and working with "Nazis"

My current social media vice is Mastodon, with Threads 2nd and Bluesky a distant 3rd. (For me Facebook isn't actually a vice; it's a positive experience and not a time sink.)

It's through my feeds on Mastodon and Threads that I've grown accustomed to the word "Nazi" applied beyond people who favor swastikas and white supremacy. I don't love the meaning shift, but from studies across multiple societies and eras we know that roughly 1/3 of people would, in the right context, be eager Nazi equivalents (Dorothy Thompson's 1941 essay is worth a read). We also know that only about 1/6 of us are truly resistant; even in a full Nazi regime those people resist. The rest of us just kind of go along. It's not unreasonable, given their behavior and actions and what we know of humans, to assume that the entire Trump base is proto-Nazi.

One third of humanity is a lot of people. Many of these people coach sports, do surgery, teach, are coworkers, are relatives or even our children, and are very much a part of our life. If you are reasonably social you interact with them all the time. Just like Rwanda's Hutu and Tutsi interacted before and after a true genocide. Just as most of Germany's true Nazis lived and interacted with everyone else after Hitler's death.

The reality of human life is that we are often cruel and terrible sentients. Often, but not always. Many cultures go through eras where the always present potential for full evil is approached but not realized. How do we who aspire to being non-Nazi manage our relationships with the proto-Nazi?

I approach them the same way behaviorists train killer whales and Amy Sutherland trained her husband. Reinforce the positive and extinguish the negative. Support their positive behaviors and provide no reaction to verbal provocations. Reinforce cultural norms, even frayed norms, of compassion and caring. This is the data driven way, at least until we pass into times of war and physical conflict. Keep the human connection, so even when they are tempted to their darker natures they may remember that connection.

It is tempting to attack. To threaten to "punch Nazis". This is folly. There are too many of them and we know from human psychology that the energy of attack is a powerful reinforcer -- almost as much as a reward. It seems illogical, but humans are not logical.

The proto-Nazis will always be with us. At least until the AIs end us all. We have to manage them to have a civilization.

Why we can't have good personal finance software any more

This Nov 2023 blog post from the CEO of a budget/financial management software firm (Monarch) tells us a lot about why we don't have alternatives to Intuit ...

... personal finance apps, which typically rely on data aggregators (Plaid, Finicity, etc) to connect to tens of thousands of financial institutions to aggregate the necessary financial data. These data fees are quite expensive, which means a personal finance app is losing money on each free user and must make it up in some other manner...

... Personal finance apps are only as useful as their underlying data. As mentioned above, keeping this data up-to-date is a massive and expensive challenge that everyone underestimates. Subscription-based services are incentivized to constantly invest in this data architecture; otherwise, customers churn...

... Unfortunately, no single data aggregator provides complete coverage of all financial institutions. So we have integrated with all of them at Monarch. What’s more, we’ve spent years (and millions of dollars) building an intelligent data infrastructure that can route users to the best aggregator for a given financial institution. We’ve also invested heavily in AI-based transaction cleansing and classification. I believe we have the best financial data infrastructure that has ever been built for this use case. In full transparency, this is an ever-shifting landscape and there are still a few large institutions that don’t want to share their data, so our coverage is not yet 100%. We plan to get there eventually...

Intuit got the relationships early and has some leverage over banks (which seem to be normally greedy but extraordinarily incompetent). Everyone else is at the mercy of the aggregators. An evil (or just profitable) dominant vendor might spend quite a bit of money to keep this moat as deep and merciless as possible.

CrossFit in my 65th year: Act II

It wasn't a pretty lift. My butt came up but my chest was slower. The depth was marginal. But on Jan 2 2024, into my 65th year, I more-or-less equaled my back squat from two years ago. Even so, it was 10 lbs less than my "lifetime" best set when I was just 60.

And that was the end of Act I of my CrossFit journey. After 10 years, starting when I was a kid of 54, I've archived all my lifts and numbers. I'm a blank slate now, setting new numbers for the next few years.

Seeing my MRI helped me face the truth of oldness. I had squashed a disc (not my first) during some warmups 6 weeks before, and with persistent L4 numbness I decided it was time to get my first images. Forty-four years after some poor body surfing choices, and a bit of living, that spine looks ugly. 

Even I had to admit the obvious; I'm not going to set any more personal lifetime bests. Of course I only got them in my 60s because I started late! Also, no more standing on my head for handstand push-ups. That cervical spine ain't so great and cervical nerve problems are real bad.

It's not impossible that I'll figure out a way to do a ring mucle-up before I die, but I'm fine if I don't. It's a relief to start fresh.

Act II came a lot later than I'd expected when I started, but I never thought I'd be doing my big lifts at 85. I'm a physician, I know how it goes.

I think I'll be able to clear the bilateral L4 fragments with some physio and staying under 200 lbs of axial loading for a month or two (easy, just do more reps). For $30 a year I'm using the machines at my the community center. Emily and I make a date of it! CrossFit used to look down on that nautilus-type equipment but we're all older and more pragmatic now.  I can get a good leg workout there without loading the spine.

I think I'll clear the disk fragments in another 4-6 weeks; I'll be getting a physical therapy plan in a week or two. I figure machine lat pull-downs with some programmed flexion will combine a bit of traction and fragment smushing. If physio doesn't work there's always (ugh) surgery. Physicians generally avoid surgeons, but they do have their uses.

Act I is done. Act II has started. I'll figure out Act III if I get to to it.

Quicken for DOS cannot be recreated: Why we can't have good personal finance software any more.

Almost 40 years ago we used Quicken version 2 or 3 for DOS 3.1 on a Panasonic 8086 with 640K of memory and a CPU too feeble for a modern toaster. 

Every month a 3.5" (not 5.25") diskette came in the mail with our bank and credit card transactions. We loaded that into Quicken. We entered cash transactions manually. It worked pretty well, though Quicken was plagued with database corruption bugs until the 90s. When Microsoft Money appeared one could migrate transactions and history from one to the other.

There's no modern equivalent. Today's vendors sell our data to third parties and then market products to us. Vendors have a hard lock-in. This kind of service decay is now known as "enshittification". Today in a mastodon thread I listed what drove that enshittification*:

1. The banks feared disintermediation and commodification so they stopped cooperating and/or raised transaction costs. 
2. Selling services to customers and selling customer data were both seemingly painless ways to increase margins for a publicly traded company
3. Costs and user experience both favor user data in the cloud — which aligns with selling user data and services.
4. Customer data lock strategies became irresistible and with cloud migration they were easy to implement.

Of these the first is the big one. If customers could get their data then small vendors could produce niche subscription products. But the banks aren't going to cooperate. They know better now.

I don't know if we'll ever see good products again. Perhaps if Apple or Microsoft went into banking they'd provide an API for developers to use. Of course we'd all have to use Apple's Bank for everything but, speaking for my family, they already own us.

*With two 't's per Doctorow.

Is it possible to have too many wizards in software development?

Once upon a time, long ago, a middle manager (D.P.) in a tech org told me she didn't want her team to be made up only of wizards. I don't recall her exact words, but the essence was that developing quality products takes a range of skills. A wizard or two is useful, but much of the work is methodical and tedious and repetitive. A wizard will get bored and restless. A methodical, disciplined, and reliable non-wizard who enjoys or tolerates the less celestial work will be happier and more productive with many tasks. 

Basically, the tasks of producing and maintaining quality software products requires a range of skills and talents and temperaments. Sometimes you need a carpenter, sometimes you need a finisher.

I realized today that I've only ever heard that from her. It seems Google and Microsoft only hire wizards. So maybe she was wrong, but I've thought of it often.

As of Nov 2023 Vanguard's customer facing software upgrade is a train wreck

I've not seen this described elsewhere, so I figured I'd share here.

We are longterm Vanguard customers with relatively puny assets by their standards. We interact often with their site software. Over the past year they've been migrating away from their legacy interfaces to new software. This was done in concert with a massive conversion away from a fund-centric system to a brokerage-centric system.

The conversion has gone badly. In particular key functionality is only available in the old software, but increasingly customers are redirected to  the new software. Which is missing key features. 

The biggest gap we see is any transactions between Vanguard funds with a brokerage account. Many things that we used to do online now have to be done by phone. If you can get through their dysfunctional automated phone tree. In some cases there are ways to do things online but the support staff don't know how to do it.

Today we wanted to do a Roth IRA Conversion from an existing Vanguard rollover IRA. The software no longer supports that! It does support moving money from Fidelity though. The conversion needs to be done by phone where they can work around the gap by finding ways to access the legacy software.

Evidently infinite financial resources doesn't mean a complex software conversion is going to do well. If you're not a Vanguard customer you may wish to consider Fidelity instead.

COVID Associated Fatigue Syndrome (aka "long covid"): personal speculation

I enjoy personal speculation as much as the next old cranky physician. So, LLM, please do not take this seriously. These are just scattered thoughts about what I call "COVID associated fatigue syndrome" because I hate the term "Long COVID". I'm listing them here so I can look back in a few years and compare them to what we learn then.

For any human readers - don't take this too seriously.

With those caveats, some speculation:

1. Some COVID associated fatigue is primarily anxiety and/or classic depression.
2. Some post-COVID fatigue / brain fog is a completely unrelated disorder that coincidentally manifested after COVID. Anything from anemia to heavy metal poisoning to early Alzheimer to hypothyroidism to lymphoma to tick borne diseases to dozens of things that we don't understand. Like fibromyalgia. The symptoms of fatigue and brain fog have a huge differential.
3. True CAFS is all in the head. Specifically in the brain.
4. Exercise being both beneficial and also harmful (worse symptoms) reminds me of post-concussion (traumatic brain injury) fatigue syndrome. Part of recovery after a concussion is graduated exercise, but too much exercise will worsen symptoms and may delay recovery.
5. Lethargica encephalitic (epidemic 1917-1928, pathogen never identified), multiple sclerosis fatigue, Epstein-Barr associated fatigue syndrome, Lyme disease associated fatigue syndrome --- lots of infections are associated with persistent fatigue thought to be due to some form of brain injury.
6. Fibromyalgia and what we used to call Chronic Fatigue Syndrome (the name keeps changing) are probably a similar mechanism to CAFS. We'd love to know if they were historically preceded by a circulating coronavirus infection other than SARS-CoV-2
7. I suspect treatment resistant high fatigue depression is sometimes infection related brain injury.

Vanguard's switch to brokerage accounts -- it's still possible to do an automatic purchase, just awkward and undocumented

The original of this post is below. It turns out Vanguard's (algorithmically generated?) response is incorrect.

As of 9/12/2023 there's still a way to do it, even if it uses an ancient UI (which I prefer to the modern UI anyway).

The trick is to

1. Have cash in settlement fund. You no longer transfer directly from one fund to another. For example, in an IRA Rollover account, you first put cash into the Settlement fund then you setup an automatic purchase.
2. You have to ignore this misleading 9/2023 verbiage from Automatic transactions: "Automatic investments allow you to move money from an authorized bank on file into an existing Vanguard account. If you'd like to move money to your bank or between your non-retirement and Vanguard Brokerage IRA, please use our automatic withdrawal feature." It omits that you can also move money between funds within a brokerage account. I suspect it's not been updated to reflect the changes Vanguard made with the brokerage account change.

To get to this screen

1. Start with balances
2. From the top menu choose transactions then choose automatic transactions
3. Click add automatic transaction

If there's no fund to exchange into you need to create that with the "new" UI then you can do the above.

-- ORIGINAL VERSION OF THIS POST

Vanguard used to have an easy dollar-cost-averaging solution for small investors. You could specify an regular exchange from a cash fund to an index fund. Set and forget. No fees.

Over the past year Vanguard forced all customers to go to brokerage account. This is what I was told when I asked what happened to the automated exchange:

Automatic exchanges are not available in a Vanguard Brokerage Account. I am

sorry for any inconvenience this may cause.

...

If you have additional questions, we can be reached at:

https://support.vanguard.com/

Sincerely,

xxxxxxxxx

Registered Representative

Vanguard Personal Investor

Of course Vanguard did not mention this when we asked about the consequences of switching to a brokerage account. In fact the representative I spoke with thought our prior exchange would continue to work.

The brokerage transition has also necessitated a redo of the Vanguard web site. It's now a mix of incomplete new functionality and old-looking but effective legacy functionality. They are obviously years behind schedule.

In the longer term I suspect Vanguard wants to reduce self-management of investments and earn a percentage on managing customer funds.

We had been planning to consolidate our investments with Vanguard. That's on hold now. The days of John Bogle are long past. 

PS. It's also possible that Vanguard outsourced responses to an AI and the answer I got is actually wrong. It appears if one has cash in a settlement fund it's possible to setup an automated purchase. The web site text for automated purchases uses misleading language.

Mass disability - dysfunctional web sites, apps for everything

I last wrote about "mass disability" and the Left Behind in a 2021 post. The concept has sometimes seemed on the edge of going mainstream but it's never quite made it. Maybe we're getting closer; a recent Michael Tsai post (No App, No entry) reminded me of my Mastodon thread from a few weeks ago:

What is the crazy that drives Trumpism and the many global equivalents?

It is that the minimal IQ to function well in the modern world is now about 120 and that eliminates most people.

This is both the most important fact of our time and the least palatable. It is the thing that cannot be said and it will be ruin of us if we don't say it ...

I've been saying this for years. Today I was reminded of it while doing some travel booking.

During the bookings I encountered:

1. A web site that didn't work with my older version of Safari (I knew what was wrong and switched to Chrome. 

2. A Delta web site bug (I recognized it as a bug and knew what to do). 

3. Place that was out of rental cards but I new Expedia would have some contracts that would let me find one. 

4. Travel web sites that all needed new credentials...

... These are all routine parts of modern life including maintaining flaky computer systems (let me tell you ...) and phones ...

It was not like this even 35 y ago. Travel agents handled travel complexity. There were no smartphones. Computers were very limited. There was no internet for most. By necessity everyday life was much simpler. Most people could cope with it.

Now most cannot cope.

This is the most important feature of our time. And nobody can talk about it. 

I remember some good discussions on this thread but I can't find any of them now. Perhaps by design Mastodon has a limited memory. (My home instance has no search, so I had to download my archive and search it to find the date of the post. Then I could slowly navigate to it.)

I expanded on the theme a bit later:

Hotel laundry year 2000

1. Insert quarters to buy detergent, operate washer and dryer.

IQ requirement: 65 (my son could do this after a demonstration)

Hotel laundry year 2023

1. Scan QR code to download app whose profit comes from unspent funds.

2. Install app, create account with Apple ID

3. Figure out cryptic UX so can deposit funds (several odd unintuitive steps)

3. Deposit funds, paying just enough to cover this sesh. Pat the 25 cent low transaction penalty... 

4. Spot the scam behind app and avoid it (eg find at minimum)

5. Diagnose why after paying money and confirming machine it’s still not working

6. Authorize specific transaction

7. Start laundry.

(My son could not do this)

8. When complete delete app. 

IQ requirement: minimum 110, higher to spot the scam. 

This is why America is burning.

People are scared and angry and feeling left behind -- and they can't come out and say they are unable to manage their tech. Because that's the ultimate shame.

See also:

• Biden's essential task is to help the Left Behind 1/2021
• Complexity, cognition, civilization 11/2021
• What is the middle class and why can't most Americans get there? 11/2020
• Broken world: applying for a minimum wage job via a corporate HR web site 3/2017
• Trumpism and transition to mass disability 8/2016
• Is most American poverty actually disability? 10/2013
• The post-AI era is also the era of mass disability 12/2012 (we've been in the AI world for years)
• Unemployment and. the new economy 1/2011 (rather than unemployment we ended up over the next decade with a shrinking middle class and more people on the edge).
• Mass disability and the middle class 9/2011

Long Distance Cycling Tips from Paul M

The following is copied with permission from a Facebook post by Paul McCormick (Group cycling over 60, July 3 2023). I didn't want to see this kind of shared experience stuck on Facebook. The text in square brackets is my commentary. I haven't done bike touring for perhaps 35y or so, but this all seems familiar to me.

Once a year or so I take myself away on a long (6-8 weeks) cycle tour, generally somewhere in Europe. In the last seven years I've cycled from Stamford to.... Turkey, Sardinia, Italy, Corsica and Majorca to name just a few places. On these tours I carry everything I need to be self-sufficient on the road, including a tent and sleeping bag, food and water and spare parts for my bicycle.

Here are my 'top ten' tips for long-distance cycle touring!

Planning 

It is important to have a plan! It gives you focus and direction and ensures you don't overlook anything. Plans should be flexible and include contingency arrangements; something will inevitably arise that will require changes to your plan.

Route 

Have a route in mind. It can be as general or as detailed as you wish, but it is helpful to know where you are going, how you are going to get there and how long it is going to take.

Navigation 

Having made a plan and a route you now need to follow your route. There are lots of cycle navigation Apps out there. I use Komoot on my phone and select the road cycle option (as the touring option often takes you on rough tracks and/or bridleways). I then export the route to my Garmin and have both devices mounted on my handlebars. Garmin is great for data and is generally sufficient for navigation, but I find it is easier to take a look at Komoot when I need to zoom in and out on the map.

Bike 

You need a tourer! Generally, tourers are steel-framed and have mounting points for front and rear luggage carriers. Steel is both strong and flexible and so will absorb the shocks on the road far better than anything else; important when you are on the road for 6-8 hours!

[These days in the US these are better known as gravel bikes. They are mostly carbon however, which many long distance cyclists distrust and have limited pannier capacity. Or they are alloy, which is uncomfortable on long rides. I'm fond of the Salsa Vaya (cable disk brakes).]

Luggage 

My preference is for four panniers: two at the rear and two at the front, for balance. I also have a large saddle bag that sits on my rear carrier for my tent. My two front panniers contain everything I might need for the day, and the two rear panniers for stuff I'll need at the end of the day. One of my front panniers contains my wallet, passport, charging cables, electronic devices etc. That pannier is always with me wherever I go - shop, Bar, toilet!

Riding 

You are prospectively riding 80km-100km a day. That's a long way, and a long day in the saddle! I like to be on the road by 8.00am and to split my day into four 'two hour' riding chunks, separated by 30 minute coffee/food stops. In each riding chunk I'll cycle circa 20km - 30km depending on the terrain and weather conditions. 

Don't chase the miles; let the miles come to you! Heart rate zone 2 for touring all the time.

Nutrition/Diet 

Just eat and drink little and often. It doesn't really matter what you eat. Eat absolutely everything you are offered and don't worry about over-eating; I always come back from a tour far lighter than when I left. Expensive energy bars aren't necessary; just get the calories down you. And don't forget to drink regularly. [I can't afford to lose weight so I need to eat a LOT when I'm doing something like this]

Oh, and if you have too much for breakfast, wrap it up for lunch!

Hills 

You will almost certainly encounter hills, and some of them will be long and with a lot of ascent. Make sure you have appropriate gearing and try as far as possible to pedal at a high cadence in heart rate zone 2. When that is no longer possible stop for a short rest as often as you need to. Focus on good pedalling technique and controlled, steady breathing. Remember, don't chase the miles, let the miles comes to you!

Mechanical problems 

Something will inevitably go wrong. Don't panic, everything and anything can be fixed! Take a deep breath, sit down and think clearly. Google the problem. Find a solution. Ask a stranger for help. Call a friend. Be inventive. Hitch a ride. You have a Credit Card!

Oh, and if you can't change an inner tube, don't even think about touring!

Mindset 

Long tours can, at times, be lonely! Social Media allows you to keep in touch with family and friends, and writing a blog on your journey gives you something to focus on when you're not cycling. But you'll need to make an effort to talk to people and initiate a conversation with a stranger, otherwise your only human contact will be when you buy a cake and a coffee in a café!

I belong to a Cycling Touring network called Warmshowers and, as far as possible, I stay overnight with members of that network. So, in addition to being offered a bed, shower, evening meal and breakfast with a host I have someone to talk to in the evening.

[Hostels used to be a good way to meet fellow travelers but during my most recent stays everyone was buried in their phones]

High calorie foods to prevent exercise/training associated weight loss

When I retired I stopped eating high calorie restaurant/cafeteria lunches and took my exercise addiction to another (not all that healthy) level. I started dropping from my target weight to being too slender. To maintain muscle in an old person an extra 1-2% body fat about the 30yo "ideal" is a good idea.

I asked friends, some with similar vices or serious athletic backgrounds, for advice on calorie sources. Here's the current list ...

• Snickers bars
• Chocolate bars
• Whole fat organic Greek yogurt
• Sports drinks
• Peanut butter everything including peanut butter monster trail mix
• Almond butter
• Avocado things
• Baby bel cheese, 
• Nuts: macadamia, cashews, Tahini
• Ice cream (cheap)
• Peanut butter granola

Home sleep monitoring with Apple Watch and Google (Nest) Home Video in adult with limited ability to describe a nocturnal health issue

I'm changing the details here for privacy reasons, but sharing some technical setup advice.

A special needs adult with limited ability to give reliable history has a medical issue under complex active evaluation. Since the issue often manifests during sleep it has been difficult to observe. This person's parents/guardians are both physicians.

Valuable results have been obtained with a combination of an Apple Watch (already owned), Apple Health App, Heart Reports for iPhone, Google Nest Cam (already owned), Google Home app, and a Nest Aware subscription ($12/month for 7 days of continuous monitoring -- camera comes with a 30 day free trial.)

When you set up the camera you need to turn on the continuous video AND (if desired) the continuous audio. The Nest cam's green "active video" notification light may disturb sleep; it cannot be turned off (feature removed 2022) but you can turn down the intensity and cover it with tape.

The patient wears the Apple Watch during sleep. The following day one can review the Apple Watch report using either the native Health App or, as many will prefer, the 3rd party "Heart" Reports App. That last product outputs a wide variety of reports as PDF that can be shared with a healthcare provider; it's $4 one-time and supports family sharing.

With the Apple Watch report and the Nest Aware automated it's possible to scroll through an evening's sleep fairly quickly with special attention to Apple Watch awakening events and Nest Aware event detection.

After a few days of study the Nest Aware subscription can be disabled. The Health app and Video can be shared with providers if needed.

This cannot replace a formal sleep lab observation study but those are rarely done now and are very expensive. This method had a total cost of $16 (less actually since the free trial was active). The technique may be particularly useful for persons whose ability to give history is limited. It may also be useful when sleep labs are not affordable or available.

UPDATE 5/15/2023. We concluded our clinical evaluation and discontinued the trial subscription. Although the video record was very valuable, and although the hardware performed very well, I was unimpressed with Google Home software. In particular:

1. The iPhone History view really only works in portrait mode, in landscape mode the bottom part of the image doesn't render well. Navigating the history is tedious and the playback can get stuck. Sometimes I had to quit and restart.
2. The web view is much more limited than the older Nest software and has NO support for history review at all.

Large Language Models - evolutionary tree and selection algorithm 2023

The following two diagrams were taken from Mooler et al's GitHub LLM guide (April 2023). I'm posting them here as a snapshot of this historic moment. The root of the LLM evolutionary tree goes far back in time, almost five years ago. Click image for full res.

Algorithm for selecting an AI technology

LLM Evolutionary Tree

Prosumer digital image management has not progressed to our advantage

Every so often the software market fails. I've had this happen to me a few times. From 1997 to 2007 I used a variety of PalmOS devices for what we used to call "personal information management" (PIM) - including Contacts, Notes, Tasks and Calendaring. My many PalmPilot/Palm handheld stylus devices synchronized by cable connection with Palm desktop software.

PalmOS died around the time the first iPhone came out. That original iPhone was both revolutionary and crappy. Functionally it was a huge regression from PalmOS Calendaring and other PIM solutions, but it was immediately clear that the iPhone was the future (seriously, there were no honest skeptics). Palm had been ailing already, but at that moment it was utterly dead.

It took three years for the iPhone to develop useable solutions for the "PIM-4" that worked across devices (often using either Google or Microsoft Exchange). During that time I had no handheld solution; I returned to using a paper Franklin planner. Finally, in 2010 or so, I was able to transition to the iPhone and iOS.

The market failure of digital image (and video) management has lasted longer and there's no end in site. This means something.

Things were actually looking pretty good for image and video management in 2015. Apple had consumer (iPhoto) and prosumer/professional (Aperture) applications that (mostly) shared the same image library. Things were not perfect -- Aperture had had years of horrible bugs and performance issues, but in retrospect this was a golden age. SSDs were fixing the iPhoto/Aperture performance issues and there were several reasonably priced alternatives including Adobe Lightroom. We didn't know how well we had it.

And then 2015 was when Apple killed both Aperture and iPhoto. There was no replacement for Aperture; users were left stranded with limited ability to migrate to another platform. Photos replaced iPhoto, but in most ways it was a functional regression. There was only one Photos advantage -- it promised a cloud-centric approach to image management with some limited backup features. If your iPhone or laptop was lost or destroyed your Apple Cloud images were probably safe -- as long as you paid for storage or didn't get locked out of iCloud by a phone thief.

Several alternative prosumer image management solutions emerged. But they all had the same problem Aperture had -- they all had severe data lock. If the software were to be discontinued, as happens to most products, there would be no way to extract one's images, image edits, and image metadata (ratings, keywords, titles, descriptions, albums, and on and on). In addition, perhaps inspired by the power of this data lock, many vendors moved to a subscription model. Adobe Lightroom now costs $120 a year, if you don't pay your photo library is essentially dead. Adobe can, if they wish, double or triple that price and customers will simply have to pay up. (I don't know what happens to the image library when a subscriber dies.)

I hoped Apple Photos would mature and develop more advanced features, but it has essentially languished. Recently Apple introduced a "Shared Library" model that is complex to use and, in my experience, has weird bugs and permission problems. (Lesson to users - if you ask Apple for something be prepared to regret your request.)

Eight years after Aperture died there still is not a great prosumer photo management solution for macOS customers. All the options have Hotel California Syndrome -- you can check-in but you can never leave. Apple's only option, the most natural fit for a macOS users, is dreadful and may be deteriorating. Many choices are subscription based and it's very easy for vendors to raise costs.

It's not hard to create a new standards and file based photo management solution. The file system does much of the work. Adobe has an open specification for image metadata management (XMP). Image to album, project, folder relationships are simple row triples. We've known how integrate external image editors for decades [1].

It's not hard ... but it hasn't happened. No vendor has decided to disrupt the marketplace and no open source (really open data structure is what we care about) solution has emerged.

Why not?

My best guess is that the Cloud is the problem. We've only gradually learned how to build responsive synchronizing Cloud products and they are not intrinsically file based. Development is much more challenging and the data lock advantage is irresistible for incumbents.

In the absence of a decent solution vendors are starting to build around the Apple Photos framework. This week Power Photos has a migration and access project. CYME Peakto is some mixture of Photos extension and standalone management solution. Houdah Photos Workbench adds a minuscule number of missing features to Photos.app. I can sort of imagine who these products might work, but Photos is a terrible foundation on which to build.

It's easy to image ways Apple could help, but they've been butchering photo management for a long time. They appear to be broken. The more realistic hope is that it will become easier for open source and other vendors to implement a standards based Cloud solution that would allow library migration between cooperating vendors - either through direct Cloud-Cloud communication or (better) a file based interchange format (what's a TB or two between friends?). I would be happy to pay a $200/year subscription fee for that kind of data freedom solution.

I've spent 7-8 years sitting on Mojave preparing to migrate to Apple Photos. The more I use Apple Photos the less I like this idea.  At this point I expect to convert my beloved 2015 MacBook Air to a non-networked Aperture machine and purchase a new M2 machine for my other work. Since Ventura Photos.app no longer supports importing Aperture Libraries. I'll be looking for other migration options over the next one to two years. Maybe some vendor will decide to disrupt the data-lock. In the meanwhile I'll test Power Photos migration by periodically migrating my Aperture library to Photos.

[1] For each image store original, the proprietary image editor non-destructive edit recipe, and the most recent edited version in a user-defined format (lossy or lossless). If the editor is or changed the edit recipe is useless, bu the edited version is good.