Yate and the P Word

Download as PDF

Back to Yate Resources

Yes the dreaded P word. Programming! If you're going to be writing actions in Yate you will be programming. Some of the things you can do are trivial while ... well, some are not.

This document will talk about the concepts and the things you should watch out for. It will not present a detailed description of every available action statement.

Put some music on, don't bite your nails and read on.

What am I working with?

Primarily you'll be working on audio files. You can write actions which work entirely on database or text files but we'll forget about that for now. The set of audio files available to you are the currently selected files in the main window, or when you're batch processing, a single folder's worth of audio files.

Within files you'll be manipulating fields. There are a lot of fields available in every file. Things such as Artist, Album Artist, Album, Title and many other metadata items are fields. The previously mentioned items are standard fields. Other field types are extensible and are user named. User Defined Text Information items, or UDTIs for short, are commonly used. You can also have named comments, (in addition to the standard Comment field), named URLs, and other less commonly used items.

What are fields? Really!

For all intent and purpose, fields are text strings. While a small number of fields are binary, there is not much you can do in actions with them, other than to delete them.

Pretty much everything that you can work with in a Yate action is a string. Strings can contain any Unicode character and are not in themselves tied to any one particular character encoding. When audio files are read and written, Yate will use the appropriate character encoding.

For you intentional programmers out there, note that there is no concept of nil. A string can be empty but it always exists. In Yate if you test a field for existence, you're trying to determine if it's empty. (Completely empty. A string with a space in it is not empty!).

Some Yate statements let you interpret a string as an integer value. When this is the case the string representation is converted to an integer value by ignoring everything in the string which cannot be interpreted from the start as an integer. If there is no valid integer that can be extracted from the string, zero will be assumed. Be aware that there is never any storage of the integer value. It is always stored as a string representation. For example the Increment statement will extract an integer from a field or named variable, will add one to the value and will then convert it back to a string for storage.

Other Yate fields expect a value of true or false. When reading these values an integer value is extract as described above. Any non zero value is assumed to be true. When setting a boolean value, true is 1 and false is 0.

Where do I store stuff?

There are three types of variables in Yate and another sort of variable type.

When an action statement refers to a Track Variable, it is referring to a field named Variable 0, Variable 1, Variable 2, ... Variable 15. These variables are fields like any other, with the exceptions that their contents are discarded when action processing is terminated and that their values are never saved in the files. When we get to execution modes, you'll see that it is very useful that files have unique fields. They enable the parellel processing of many things that you can do on audio files. Oops, another P word!.

Another type of variable is a Named Variable. Named variables can be named anything you like. Named variables are not associated with any one audio file. They are application wide and also only exist while an action is running. There are many action statements which set the value of a named variable. When you want to read the value of a named variable, you typically use an escape sequence. We'll come back to escape sequences later. As the escape sequence for named variables uses a > character as a terminator, it's a good idea to not place > characters in the name. If you reference a named variable which has never been set, its value will be the empty string. As actions are modular and might eventually be called by others, it is a good idea to initialize your named variables as opposed to assuming they will be empty.

There are ten system variables numbered 0 to 9. System variables do not get removed when you close Yate. Whatever you put in them sticks around until you get rid of them. They are really only useful if you wish to keep some persistant information hanging around. You can examine the values of the system variables outside of actions in the application preferences System Variables panel. These variables are useful for keeping notes as the viewer in preferences can display a multi line display.

As the application has evolved and more complicated actions got written, it became apparent that a better method of retaining settings for actions was necessary. This leads us to....

Action Runtime Settings

Action Runtime Settings are similar to Named Variables. However, they are not intended to be used as general purpose variables but rather a means of keeping state for an action after it terminates. Action Runtime Settings persist until they are removed. Action Runtime Settings cannot contain multi line data and in fact are truncated at the first, if any, newline character. The writing of an Action Runtime Setting is less efficient than storing a Named Variable. While Action Runtime Settings can be treated as Named Variables, it is not recommended that you do so. While they are persistent as are System Variables, Action Runtime Settings are symbolic and provide a well defined interface specifically designed for action settings. The settings can be viewed and manually edited in the application preferences Audio panel.

Escape Sequences

Escape sequences are your friends! Proper use of escape sequences allows you to perform an operation in a single statement as opposed to achieving the same effect with multiple statements. They are also the only way to access the content of named variables, system variables and action runtime settings. You can also access the initial value of fields before they were changed, properties and other useful content. If you wanted to set the Genre field to its current value with the contents of the Mood field and the contents of Variable 1 prepended, you can do it with a single Set statement and escape sequence.

Text fields in action statement definition panels have context menus to ease the insertion of escape sequences. There are many of them. There is also a context menu which essentially keeps track of the names of all named variables used in any open action ... and another which keeps track of all defined and referenced action runtime settings.

Execution Modes

Yate's execution modes are the toughest concept to grasp and possibly the most important. The two execution modes are stepwise and grouped.

A Yate action cannot execute without selected audio files. When executing stepwise, every statement is applied to each selected audio file. You can think of each file being processed in parallel. If you are executing stepwise and you issue a Copy Album to Variable 1 statement, you will be copying each selected file's Album field to its unique Variable 1 field. One statement does the same thing with possibly different data, for each selected file. If a particular statement executed stepwise does something non standard, it will be described in its documentation. When action processing starts, the initial execution mode is stepwise. Some action statements, such as those that perform artwork retrieval, are more efficient when run stepwise as Yate can ensure that any one artwork item is downloaded or loaded from the filesystem, only once.

Many actions can be written and designed to be run entirely in stepwise execution mode. However, there are times when it is required to run blocks of code as a single entity for a single file before proceeding to the next file. When a section of code is run in its entirety for a single file, you are executing grouped. Typically grouped execution is required when conditional statements are used to control program flow. When making conditional decisions, different outcomes may be required for different files. The only blocks of code that can be run grouped are actions.

If you want to force an action to be executed grouped, you can do so by placing a Force Grouped Execution statement at the start of the action file. Use of a Force Grouped Execution statement is the only means of executing an action called directly from the UI grouped. Remember, action execution always starts stepwise.

Actions can be run from the UI or via the Batch Processor. When run via the Batch Processor, the action will be run on every folder which contains audio files. When starting the Batch Processor, the root folder is specified.

Action Structure

Actions can be separate entities, effectively files, contained in the Action Manager. These actions can be called from any other action. Actions can also be inline, contained inside an action file. Inline actions can only be called from within the action file which contains them. Unless otherwise directed, an action executes from its first statement to the end of an action file or until a Start statement is encountered. Start statements are used to specify the start of an inline action. There is no end statement for an action. You either hit the end of the file, or a Start statement.

So "I have a complicated action. Should I use inline actions or create multiple separate actions?" Our general practice is that if an action can be reused, create it as a separate entity. If the action can be thought of as being used only within a specific action, create it inline. Most of the actions we supply are delivered as a single action file.

When an action is run through the Batch Processor, a search will be performed for two inline actions in the action file being run. The Start Batch inline action, if found, will be executed prior to the processing of any folders. The Stop Batch inline action, if found, will be executed after the last folder has been processed.

Flow Control - Conditional Execution

Actions execute linearly, one statement after another, unless another action is called or an if statement is executed. There is no limit to the number of actions which can be called, but recursion is not permitted. Any given action, file based or inline, can only be open for execution once, at any given time. There is also no limit to the number of nested if-else-endIf constructs.

There is only one thing that can be tested by an if statement and that is the Action Test State. The Action Test State contains a simple true or false value and is set by many action statements to report a successful or failed operation. There are conditional test statements to compare things as text, integers or dates. These statements all set the Action Test State. These conditional test statements can Set, And or Or the test state. This is an efficient means of forming compound tests without requiring multiple if-else-endIf constructs. A statement which Ands the test state is only executed if the test state is currently true. A statement which ORs the test state only executes if the test state is currently false. A statement which Sets the test state always executes. Remember, statements are executed one after another. There is no precedence in Yate statement execution.

Typically when you use more than one statement to create a compound test you will be executing grouped. This is due to the fact that the test may produce different results for each file. Three test statements (Compare Date Test, Compare Numeric Test and Compare Text Test) support the saving of the test result to a track variable as well as to the action test state. The Set, And, Or semantics are preserved when setting the track variable. This feature allows you to preserve individual test results on a per file basis while executing stepwise. After running any combination of these statements you can have a track variable set to true or false. Lots of other statements also set the action test state. You can use the Logical Set statement to combine their results with track variables set by the three statements which support track variable setting. You'll see in the next section that there is a Run statement which lets you execute an action grouped but only for those files which have a specific track variable which is true.

Flow Control - Running Actions

The Run statement is the simplest way to call another action. The statement can call inline actions and other action files. When an action is run it inherits the current execution mode unless otherwise directed. If grouped is specified, the designated action will always be run grouped. You'll note that there is no stepwise option. That's because you can never force stepwise execution. You can never run an action stepwise when you are already executing grouped. The grouped option does not change anything when you are already executing grouped. However, in our opinion, it is good practice to specify the grouped option when you want to run the specified action grouped even if already running grouped. It makes things a little clearer in the code.

There will be times when you want an action to operate on data completely outside of the selected audio files. You might be operating on data in a database file or on data stored in lists (later). In this case you do not want to run stepwise over every audio file and yet you do not want to run successively grouped on every audio file. The once option can be specified along with grouped. The effect is that the action will be run grouped but only once with a single audio file. If the choice of which audio file is active is pertinent, you shouldn't be running once. There is no way to execute statements in Yate without having at least one audio file active.

A much rarer run option is indirect. When running indirect, the name of the action to be run is determined at runtime. This is useful when you want to run one of a number of possible actions dependent on certain conditions. The field defining the indirect action name should contain at least one escape sequence for a track or named variable which contains all or part of the action's name.

The Run statement can also be told to run only for those files which have a track variable with a value of true. This is useful when a compound test has saved individual test results to a track variable. The semantics are what you would expect for both stepwise and grouped execution modes. Regardless of the execution mode, only those files with the specified track variable having a value of true will get processed. Changing the control track variable in the called action will have not change which files get processed.

The Test statement is a run variant which lets you test the action test state, optionally run an action with all the options available on the Run statement and then continue, exit or stop action processing. The statement was originally the only means of testing the action test state. In the early days of Yate, the Test and Run statements were the only flow control statements... if-then else came much later.

Actions can be repetitively executed. In its simplest form a Repeat Forever statement will continuously call an action until an End Repeat statement is executed. The Repeat Forever statement can only run inline actions.

A second repetitive execution statement is Repeat With which lets you execute an action based on components of a list. The statement calls the action once for each list element. Once execution starts, the effective list is immutable. As an example you can choose to execute an action for each unique artist extracted from all selected files' Artist fields, treating items separated by commas in the individual Artist fields as separate items. There will be more on lists later. When executing stepwise, the action will be run once for every list element. When executing grouped, the action will be run once for each selected list element on each selected file. As with the Run statement, the Repeat With statement can be executed grouped once. In this case the action will be run once in grouped mode for each list item...but only once. As with the Run statement, this is useful when you want to run grouped but you're not working on data in an audio file.

A third repetitive execution statement is Repeat For which is analogous to a typical programming language's for loop. The statement is given a start value, a stop value, a comparison operator and and an adjustment value. The execution options are the same as for the Repeat With statement.

The last means of running an action is a special case statement where you want to load an audio file outside of the set of selected files being processed. The Load and Run statement, loads the specified audio file and executes an action on that file. The action can be inline and can be specified indirect. However the execution mode is always inherited. You cannot specify grouped and once is always implied. When the action finishes, the audio file will be closed. Note that the action must include a Save statement as changes are not automatically committed when the action is terminated.

Flow Control - Exiting an Action

As already mentioned, actions execute until the end of the action file or until a Start statement is hit. There are a variety of action statements which can be executed to exit an action. The flow control effect of the exit depends on the statement. The following return statements are supported:

Exit
This statement returns from a single instance of an action. When executing grouped, execution of the action will continue on the first statement of the action with the next selected file. When executing stepwise, or if there are no more files to process grouped, control will resume at the first statement following the calling statement. This statement most clearly mirrors a typical return statement in higher level programming languages. This statement mirrors what effectively happens if you execute a Start statement.

Exit Grouped
This statement stops the execution of all actions currently executing grouped. This means that you might be returning from more than one action. When executing stepwise, the statement is effectively identical to Exit.

Next File
When executing grouped, the action immediately moves on to the next file to be processed. This means that you might be returning from more than one action. When executing stepwise, the statement is effectively identical to Exit.

Exit Repeat
Regardless of the execution mode, if the currently executing action is a Repeat Forever, Repeat For, or Repeat With statement, all repeating will be stopped for all files. When not repeating, the statement is effectively identical to Exit.

Break
Break stops the execution of the current action, regardless as to how it was called or whether the execution mode is stepwise or grouped. The next statement to be executed is always the statement immediately following the caller.

Stop
With one exception this statement stops all action processing. The exception is if you are batch processing. If you are executing the Batch Start inline action, control will proceed to folder processing. If you are processing folders, control will proceed to the next folder. If you're processing the last folder, control will proceed to the Batch End inline action.

Stop Batch Processing
This statement is effectively a Stop statement when not batch processing. When batch processing, it differs from Stop in that if processing folders, control will proceed immediately to the Batch End inline action.

Cancel
Regardless of the execution mode or where you are in a batch processing sequence, this statement stops all action execution. It is equivalent to hitting the Cancel button in the UI. Warning: modifications to open databases will not be automatically be saved when Cancel is executed.

All of the above return variants can be executed always or only when the action test state is true or false.

Exit if true

is the same as:

if true
    Exit
endif

Flow Control - Muddying the Waters

We've discussed the execution modes, conditional execution and running actions. There are a few other means of changing how statements are executed.

As already stated, the Force Grouped Execution can be used to start an action in grouped execution mode. There are times when you have an action which doesn't use any of the loaded audio files. Perhaps you are reading a database file and indirectly making changes. The Constrain Execution to a Single File statement ensures that only one audio file is visible to the action. Note that once executed, there is only one effective file until action processing terminates. Remember, actions cannot be executed without at least one selected audio file. This statement ensures that you really don't have to think about execution modes for the remainder of the action. It does allow you to stay in stepwise mode if you wish. Unlike the Force Grouped Execution statement, Constrain Execution to a Single File does not have to be the first statement in the file. It can be executed at any time.

There are times when you only want to process certain files based on a condition. However, you might want to perform a function which is only valid stepwise, such as Renumber Tracks, on those files. The Ignore Files statement limits the number of files which are visible to the action, based on a track variable being true or false. When files are ignored, they will not be processed by action statements. You can successively ignore files if you wish. The Restore Ignored Files statement is used to make everything visible once again.

Lists

List processing is one of the more powerful features in Yate. The nature of tagging is such that you are quite often working on lists. For example, lists of artists, or composers, etc.

A list in Yate is simply an interpretation of a field or named variable. It is not a data type in and of itself. A list is interpreted based on its implied or supplied delimiter. Lists can be enumerated by the Repeat With statement. You can search, add to, remove from, build and count the number of items in a list. You can get, set and remove individual list elements by index. Further, you can combine lists by their union, intersection, exclusion and by filtering. You can have lists of lists. Lists are the closest Yate concept to a high level programming language's array. You can also convert a list to and from CSV formats.

While lists are typically analogous to arrays, Yate supports the concept of key-value lists which are effectively a high level programming language's dictionary. In a key value list each list element has a key and an associated value. You can have values which are other lists or other key-value lists. All of this is possible by the selective choice of delimiters for each contained list.

Remember that lists are always simply strings in their implementation. You can mix and match list and text statements on the same data.

The Build List statement is used to construct lists. The source can be the contents of fields found in audio files or a named variable. When building lists based on the contents of fields, you can elect to only process those audio files which have a track variable with a value of true. This is analogous to the equivalent feature in the Run statement.

Text Files

Yate can read and write text files. You can use statements to convert CSV databases to lists and convert lists to CSV data to be written.

Using Text statements such as Scanner you can parse virtually any text file. The Repair Files from Audio File Health Check Log action parses the plain text file produced by the Audio File Health Check application. Virtually any text file containing metadata can be read and applied to specified files.

Query Databases

A database file, (CSV or tab delimited), opened as a Query Database, is an extremely efficient means of manipulating large amounts of data. You can access individual cells, rows and columns. You can perform successive filters to effectively produce a list of database references which you can then enumerate and operate on.

You can write actions to dynamically load files referenced in a database by the Load and Run statement and then use Query Database statements to modify the file based on data in the database.

Editing States

Yate maintains three versions of the metadata in a file. The initial state represents what was in the file when first loaded or after being saved. The current state represents the current, possibly unsaved, metadata. The editing state represents the metadata at a particular point in time (think snapshot).

In the UI the editing state can be manually and automatically updated. Also you can restore to the editing state as opposed to the initial values.

In actions you can use the Update Editing State and Restore to Editing State statements to maintain and restore to snapshots of the metadata.

An interesting point to remember is that while the initial state is reset when a file is saved, the editing state is not. It is therefore possible to revert to a state before files were saved.

Debugging

The only real method of debugging an action is to use the old school method of printing things. While you can use any statement in the Prompt family, the Dump Variables statement was designed for this purpose. You can pick and or construct (with escape sequences) a message to be displayed. You can display a snapshot of the sixteen track variable fields or display the track variables for every file active in the current execution mode. You can display all named variables and the system variables. You can display a call stack and the current value of the action test state. Phew... Make sure you set the UI option if you're debugging an action run through the Batch Processor, otherwise the data will go into the log file.

Action statements can be disabled and re-enabled from the context menu in an Action Editor window. This might make things a little easier while debugging as you can leave debug statements in the file until you are done.

Summary

We know that there are a few strange programming concepts here. However, they are really necessary to implement a diverse range of actions from the trivial to the extremely complicated.

Additional information, including sample actions can be found at Yate Resources. You can also visit the forum.

Don't stress, take your time and play around....while listening to some calming music.









History

Release 1.0   2016-09-07
First release.
Release 1.1   2017-05-03
Added information on the Repeat For statement.
Release 1.2   2017-10-17
Described the options available on all return statements. Exit, Cancel, etc.
Release 1.3   2017-12-10
Described the ability to save test results to variables and to run grouped only on files if a specific variable is true. Fleshed out some other sections.
Release 2.0   2018-04-03
Updated for changes in Yate v4.
Release 2.1   2018-04-25
Updated for changes in Yate v4.1
Release 2.2   2018-05-22
Minor changes.
Release 2.3   2018-09-13
Updated to discuss a few additional flow control statements.

Copyright ©️ 2016-2018 2ManyRobots.  All rights reserved