You're not wrong. It's just bad design ... and it's been that way since betas of Catalina.
There seems to be an internal lock in the Apple App but it doesn't always seem to work. I've had Music get into a state where it cannot be closed until AMPLibrary settles down. One would image that that's the way it should work.
Using a background process to insure the integrity of your application's data is fine... however then silently losing everything because the application closed is not so fine. It's also not so fine to eat 100% of the CPU.
There are lot's of other cases like this with Music. When an album is added to Music, Yate waits after every track is added for the acknowledgement that everything worked. At that time it gets a PID. However, there is a timing window, if Music has just opened, that the PID returned is in fact discarded by Music and another is internally issued ... or not.
I don't expect any of this to be fixed. Sadly none of these issues seem to matter anymore. 10.14 had a bug where a port was lost after every Apple Event issued on a background thread. This bug was around since 10.14 betas and was only fixed in 10.15.
All of this has been reported, numerous times, but I can't remember the last time a bug was fixed. 10.15 had a bug where file save panels would internally crash if you were saving a non standard Apple filetype, (such as m3u or cue), and the file already existed. The application would soft crash and the user could not be notified ... and the existing file was not replaced. I reported it a number of times and used paid developer accesses to try to get it fixed. At one point the bug was marked as closed (fixed). Turns out that it wasn't and I had to use more paid credits to get it re-activated. I sent them sample code, detailed documentation and a video ... and it still took months to get anyone to even look at it.
The point of all this ranting and raving is that regardless of what AMPLibraryAgent is doing ... we'll have to live with it.
There are lots of potential fixes for AMPLibraryAgent on the web ... some of them amusing.