We start our return to semi-regular writing with a small-but-awesome custom function: `Pop`. If you grew up in the American mid-west like me you might think I’m talking about [soda|soda pop|coke|soft drinks]. But I’m actually talking about [stacks][pop]. In the computer science world, `pop` means to take something off the top of a list, and it is exceptionally handy to make it easy in FileMaker scripting.
[pop]:http://en.wikipedia.org/wiki/Stack_(data_structure)
## Looping Through Lists
Imagine you have a list of values. (I know, what a weird concept in a database).
Isabel
Sophia
Mamie
Geoff
Jesse
And you want to do something with these items in a script. Typically, for example, you might want to make a new record for each item in the list. You probably already know that FileMaker has several functions designed to make working with return-separated lists easier. For instance, suppose the list above was in a variable called `$names`. Here are some functions you might use, and their results:
GetValue($names, 3) => returns “Mamie”
LeftValues($names, 2) => returns “Isabel¶Sophia¶”
MiddleValues($names, 3, 2) => returns “Mamie¶Geoff¶”
RightValues($names, 2) => returns “Geoff¶Jesse¶”
Writing our script using these functions isn’t particularly difficult. Here’s the form we normally use:
# assume $names has a return-separated list of names
# …
Loop
Exit Loop If [ValueCount($names) = 0]
Set Variable [$name, GetValue($names, 1)]
New Record/Request
Set Field [The Name Field; $name]
# set other fields, do more with $name, or whatever*
Set Variable [$names, MiddleValues($names, 2, 999999)]
End
> Note: You might wonder whey I have the `Exit Loop If` step right after the start of my loop. I do this a lot. It means my loop won’t run even once if the $names variable happens to be empty, which saves me an `If` step around the loop.
##Pop: For the Lazy Programmer In Your Life
We write scripts like this a lot. We pull the first item out of the variable, use it or store it, and then re-set the variable to remove that first item again. Although it is minor, this is non-ideal for a few reasons:
1. The code is kind of strange. What does this code do? Why all those magic nines?
2. Any time you write the same couple of lines over and over again, your inner-time-saver-alert should start buzzing.
3. Call me lazy, but I’m always happy to save a few clicks and/or keystrokes.
And so, we wrote `Pop`. Here it is:
Pop(variableName) :=
Evaluate(
“let([
top = getValue( ” & variableName & ” ; 1 );
” & variableName & ” = middlevalues( ” & variableName & ” ; 2 ; 999999999999999 )];top)”
)
In a nutshell, this function takes the *name* of a variable as its parameter. It returns the first value in the variable *and* removes that value from the variable all in one step.
>Note: Pay attention to the previous sentence. The parameter you pass to `Pop` is the *name* of a variable, not the variable itself. You want this: `Pop(“$variable”)`, not this: `Pop($variable)`.
For the curious, I’ll explain how it works in a moment. But first let’s see it in action. We can now re-write our loop like this:
Loop
Exit Loop If [ValueCount($names) = 0]
Set Variable [$name, Pop(“$names”)]
New Record/Request
Set Field [The Name Field; $name]
# set other fields, do more with $name, or whatever
End
Instead of using `GetValue` to get the first item in the list, and then removing it later, we do it all in one step using `Pop`. In fact, since `Pop` is a *function* instead of a script step, we can use it right inside other expressions, so we can simplify our loop even more:
Loop
Exit Loop If [ValueCount($names) = 0]
New Record/Request
Set Field [The Name Field; Pop(“$names”)]
# *set other fields, do more with $name, or whatever*
End
This time, we get the value, remove it from the variable, and put it in the field all in one step. As a function, `Pop` can be used in other places too, like inside larger expressions:
Substitute(
$something;
[Pop(“$to_remove”); “”];
[Pop(“$to_remove”); “”];
[Pop(“$to_remove”); “”]
)
Or in recursive custom functions:
If (ValueCount($whatever) = 0; “”; Pop(“$whatever”) & AmazingFunction
It is often really handy to be able to combine fetching and removing into one tiny expression.
## How It Works
The `Pop` function uses an oft-overlooked FileMaker power-function called `Evaluate`. This is another one of those *meta-*features in FileMaker. `Evaluate` lets you process calculations inside your calculations.
When you write a calculation, you combine functions, values, fields, and operators to produce some useful result. Normally that result is some new data to display or use in your scripts. As you no-doubt know, FileMaker’s calculation engine provides some very powerful tools to process input and produce interesting results.
When you use `Evaluate`, the *result* of one calculation can be a calculation formula itself. Here’s a trivial example:
Evaluate(“1 + 2”)
In this case, the parameter we’re passing to `Evaluate` is the text value “1 + 2”. Since this is a valid FileMaker calculation, `Evaluate` will, well, evaluate it, and the final result of the formula will be `3`.
Of course that example is silly. If I wanted to add `1` and `2` in a calculation, I’d just write it like this, and save a lot of trouble:
1 + 2
But since this is a calculation, you can get as complicated as you want:
Evaluate(Choose($something, “Average”, “Min”, “Max”) & “(1, 2, 3)”)
That equally-ridiculous formula will return either an average, minimum, or maximum of the values `1`, `2`, and `3` depending on the value of `$something`.
*But why?! Why would I ever do that?*
Well, you wouldn’t. But using those techniques, you can do all kinds of other cool things, like, oh, for instance, write `Pop`. All `Pop` does is generate a new calculation formula that looks like this:
Let(
[top = GetValue($my_variable, 1);
$my_variable = MiddleValues($my_variable, 2, 99999999)];top
)
In other words, it makes a calculation that removes the first item from a variable and returns it. Since we don’t know the name of the variable ahead of time, we have to work the variable name into the calculation *using a calculation*. (I know, meta…). Once we’ve built this little formula, we use `Evaluate` to run it for us.
Easy as pie (wrapped in cake, wrapped in an enigma).
## Why Would I Ever Use This
If you’re like me, right now you’re thinking this is a lot of work for a very little payoff. It saved a line or two in my loop. But I encourage you to try it out the next time you need to process lists of values. Like all custom functions, it is easy to abstract the complicated part away in your database and never worry about it again, calling on it here and there with an easy name. And that, truth be told, is the core power of programming.
Welcome Back, Geoff. It is great to see you writing again.
Thanks for inflecting a bit of CS into FileMaker. One comment, why use the “999999” instead of “Valuecount ( variablename) – 1”?
Don:
Mostly because it is habit. Long ago, I tested the performance difference between the 9’s and `ValueCount`, and it is negligible (at least in 7+) but it is easier to type and requires less thought. On the other hand, it is harder to read than `ValueCount` so I could see an argument against as well.
Geoff
LeftValues, MiddleValues and RightValues will always end their result with a carriage return (even if it’s not there).
LeftValues ( $names , 2 ) => returns “Isabel¶Sophia¶”
MiddleValues ( $names , 3 , 2 ) => returns “Mamie¶Geoff¶”
RightValues ( $names , 2 ) => returns “Geoff¶Jesse¶”
Good to know, specially when you’re parsing data 😉
Hello Geoff,
I’ve always been a counter guy for loops so I never thought to use a pop method like this (in FileMaker anyway). I got curious and did a little performance test to see which is fastest. I wrote 3 scripts that iterate through a list of 25,000 values and set a field to the next value until the end of the list is reached. Here are the results:
Counter Method: 23 seconds
Pop In Loop Method: 48 seconds
Pop Custom Function Method: 52 seconds
But… the question is, how often do you need to loop through 25,000 values? When I test lists of 1,000 and 5,000, the difference is minimal. Your Pop custom function is really easy to use and keeps the script very simple, I like it!
Here’s a copy of my test file if anyone is interested:
http://www.scodigo.com/files/PopOrCounter.zip
Regards,
Micah
Tatai:
Absolutely correct. Thanks! Fixed.
Geoff
Micah:
Thanks for the update. Interesting results. As you say, though, 25,000 is atypical. One of my favorite quotes is “Premature optimization is the root of all evil.” 🙂 I tend to favor easy to write and/or easy to read over faster in most cases until I find a performance problem. And I don’t think we’ve ever had a pop-related performance problem in practice.
I fully get that “easy” is subjective though, and your point is valid. Something to keep in mind.
Thanks!
Geoff
Excellent article, Geoff. I’m with you in favoring easy to read and write (i.e. easy to maintain) solutions. I suppose it’s somewhat in the eye of the beholder, since after reading this I will probably still use something like:
Set Variable[ $i ; $i+1 ]
GetValue( $names ; $i )
… unless of course I need to dwindle the list for some reason. Then I’ll head straight back here!
Hi Geoff,
Great post – and fantastic function.
I’ve (embarrassingly) missed something obvious along the way, because the scoping of $names is a real eye opener for me.
I’ve always assumed that anything a function did to a local variable passed to it would be confined to the function’s space, and not have any effect on the original variable itself once the function was finished. The ability to have a function operate directly on a local variable passed to it opens up a lot of new (to me : ) possibilities.
Thanks!
Brian
Tom:
Understood. I think it is a matter of style and the particulars of the application at hand. I use your method too sometimes.
Brian:
Right there with you. I was surprised too when I first discovered this. I thought
$variableswere “local” and$$variableswere “global.” But the better terms are “script variables” and “global variables.”$variablesare scoped to the currently running script, and anything you do to modify them impacts the variable in that script. When the script ends, the variable goes out of scope, and a subscript will start a new scope, and restore the old one when done.This has two interesting implications:
1: You should generally avoid usine
$variablesin your custom functions unless you really intend to modify script-scoped variables. Use this:Instead of this (which I see now and again):
2: Calculations that run outside a script can define script
$variablesthat live at the top of heap, so to speak, and are in existence when no script is running. Some people use this fact sometimes, although I find it so strange I avoid it from sheer terror. 🙂Geoff
Geoff:
Thanks for the excellent follow-up – that’s by far the clearest explanation of variable scoping in FM that I’ve heard.
I’ll join you in steering clear of #2. The idea of seemingly “local” variables nebulously lurking around in my databases hurts my brane : )
Best,
Brian
man, I never thought of setting the $variable in a let() function. I could have saved a few steps that way.
Hello, Geoff,
I just found out you were back with your insightful (and rather amusing at times, I should add) posts, and that made my day! Welcome back.
Hello, almighty all, may this message find you productive and at peace.
Geoff, I used to declare local and global variables in Custom Functions as the result of a calculation, such as this,
Not long ago, I received a remark from a forum user stating that (sic) “Declaring persisting variables during the run of a custom function is extremely problematic: if you have two calculation fields using the same CF, they will overwrite each other’s variables.”
I took his advice, and decided to add an extra line of code to the calculations. Using the example above, it would then be,
I believed this change would solve the problem; nevertheless, another user kept insisting this is “bad practice.”
I rely on local and global variables inside custom functions for two reasons:
1. to pass the results into fields or as scripts parameters;
2. to check, using calculation fields, for errors.
I know the over-usage of variables might add to memory heap; but, besides that, why their usage inside Custom Functions could be considered “bad practice?”
@ Micah and Geoff:
Atypical situations do exist; no doubt about that.
As I write this message, my copy of FileMaker is performing a Script that compiles data from 5565 cities — one related to the other 5564. The data compiled is:
• Geographical (city’s name, state, region, coordinates, altitude, etc);
• Distance between main and tested cities in Mi and Km (from coordinates);
• Boolean if tested city falls into main city’s range of 500 Mi.
After a five-day period, FileMaker is about 55% done.
This is despairing. 🙂
God Bless and all the best,
Edward
Great article.
I notice you explicitly set the target field to the $variable
However if you add $name as an auto entry calc in the target fields define fields options –
For setting a single field a script set is fine sometimes however when I am setting several fields after the New Record each field that gets data past to it during a script will have an auto enter calc with a script variable with a variable named similar to the field.
Declare variables by setting a $someVar to=
Go to target layout.
Create new record
All fields are filled in, and creating fields outside a this script no data is auto entered in since the $name or $uid is not currently scoped.
Cheers
Stephen