کاروان ها در سکوت و دزدها در قال و قیل

سال ها این دشت نشنیده است آوای رحیل

کوچ کردن قصه شد، مردانه ماندن یک خیال

کودکان افسانه می بافند از مردان ایل

بوی رخوت راه های دشت را پر کرده است

روز باران گله را خوابانده چوپان در مسیل

غرق در افسون فرعونی کسی راهی نشد

پا به پای اعتقاد هیچ موسایی به نیل

بر زمین تا چشم می بیند مترسک کاشتند

در فراسوی زمان مُردند مردان اصیل

انتهای هیچ راهی منزلی پیدا نبود

جاده ها آزرده از یک امتداد بی دلیل

(Learn JavaScript Higher-order Functions, aka Callback Functions)

In JavaScript, functions are first-class objects; that is, functions are of the type Object and they can be used in a first-class manner like any other object (String, Array, Number, etc.) since they are in fact objects themselves. They can be “stored in variables, passed as arguments to functions, created within functions, and returned from functions”¹.

Because functions are first-class objects, we can pass a function as an argument in another function and later execute that passed-in function or even return it to be executed later. This is the essence of using callback functions in JavaScript. In the rest of this article we will learn everything about JavaScript callback functions. Callback functions are probably the most widely used functional programming technique in JavaScript, and you can find them in just about every piece of JavaScript and jQuery code, yet they remain mysterious to many JavaScript developers. The mystery will be no more, by the time you finish reading this article.

Callback functions are derived from a programming paradigm known as functional programming. At a fundamental level, functional programming specifies the use of functions as arguments. Functional programming was—and still is, though to a much lesser extent today—seen as an esoteric technique of specially trained, master programmers.

Fortunately, the techniques of functional programming have been elucidated so that mere mortals like you and me can understand and use them with ease. One of the chief techniques in functional programming happens to be callback functions. As you will read shortly, implementing callback functions is as easy as passing regular variables as arguments. This technique is so simple that I wonder why it is mostly covered in advanced JavaScript topics.

What is a Callback or Higher-order Function?

A callback function, also known as a higher-order function, is a function that is passed to another function (let’s call this other function “otherFunction”) as a parameter, and the callback function is called (or executed) inside the otherFunction. A callback function is essentially a pattern (an established solution to a common problem), and therefore, the use of a callback function is also known as a callback pattern.

Consider this common use of a callback function in jQuery:

//Note that the item in the click method's parameter is a function, not a variable.
//The item is a callback function
$("#btn_1").click(function() {
  alert("Btn 1 Clicked");
});

As you see in the preceding example, we pass a function as a parameter to the click method. And the click method will call (or execute) the callback function we passed to it. This example illustrates a typical use of callback functions in JavaScript, and one widely used in jQuery.

Ruminate on this other classic example of callback functions in basic JavaScript:

var friends = ["Mike", "Stacy", "Andy", "Rick"];

friends.forEach(function (eachName, index){
console.log(index + 1 + ". " + eachName); // 1. Mike, 2. Stacy, 3. Andy, 4. Rick
});

Again, note the way we pass an anonymous function (a function without a name) to the forEach method as a parameter.

So far we have passed anonymous functions as a parameter to other functions or methods. Lets now understand how callbacks work before we look at more concrete examples and start making our own callback functions.

How Callback Functions Work?

We can pass functions around like variables and return them in functions and use them in other functions. When we pass a callback function as an argument to another function, we are only passing the function definition. We are not executing the function in the parameter. In other words, we aren’t passing the function with the trailing pair of executing parenthesis () like we do when we are executing a function.

And since the containing function has the callback function in its parameter as a function definition, it can execute the callback anytime.

Note that the callback function is not executed immediately. It is “called back” (hence the name) at some specified point inside the containing function’s body. So, even though the first jQuery example looked like this:

//The anonymous function is not being executed there in the parameter. 
//The item is a callback function
$("#btn_1").click(function() {
  alert("Btn 1 Clicked");
});

the anonymous function will be called later inside the function body. Even without a name, it can still be accessed later via the arguments object by the containing function.

Callback Functions Are Closures
When we pass a callback function as an argument to another function, the callback is executed at some point inside the containing function’s body just as if the callback were defined in the containing function. This means the callback is a closure. Read my post, Understand JavaScript Closures With Ease for more on closures. As we know, closures have access to the containing function’s scope, so the callback function can access the containing functions’ variables, and even the variables from the global scope.

Basic Principles when Implementing Callback Functions

While uncomplicated, callback functions have a few noteworthy principles we should be familiar with when implementing them.

Use Named OR Anonymous Functions as Callbacks
In the earlier jQuery and forEach examples, we used anonymous functions that were defined in the parameter of the containing function. That is one of the common patterns for using callback functions. Another popular pattern is to declare a named function and pass the name of that function to the parameter. Consider this:


// global variable
var allUserData = [];

// generic logStuff function that prints to console
function logStuff (userData) {
    if ( typeof userData === "string")
    {
        console.log(userData);
    }
    else if ( typeof userData === "object")
    {
        for (var item in userData) {
            console.log(item + ": " + userData[item]);
        }

    }

}

// A function that takes two parameters, the last one a callback function
function getInput (options, callback) {
    allUserData.push (options);
    callback (options);

}

// When we call the getInput function, we pass logStuff as a parameter.
// So logStuff will be the function that will called back (or executed) inside the getInput function
getInput ({name:"Rich", speciality:"JavaScript"}, logStuff);
//  name: Rich
// speciality: JavaScript

Pass Parameters to Callback Functions
Since the callback function is just a normal function when it is executed, we can pass parameters to it. We can pass any of the containing function’s properties (or global properties) as parameters to the callback function. In the preceding example, we pass options as a parameter to the callback function. Let’s pass a global variable and a local variable:

//Global variable
var generalLastName = "Clinton";

function getInput (options, callback) {
    allUserData.push (options);
// Pass the global variable generalLastName to the callback function
    callback (generalLastName, options);
}

Make Sure Callback is a Function Before Executing It
It is always wise to check that the callback function passed in the parameter is indeed a function before calling it. Also, it is good practice to make the callback function optional.

Let’s refactor the getInput function from the previous example to ensure these checks are in place.

function getInput(options, callback) {
    allUserData.push(options);

    // Make sure the callback is a function
    if (typeof callback === "function") {
    // Call it, since we have confirmed it is callable
        callback(options);
    }
}

Without the check in place, if the getInput function is called either without the callback function as a parameter or in place of a function a non-function is passed, our code will result in a runtime error.

Problem When Using Methods With The this Object as Callbacks
When the callback function is a method that uses the this object, we have to modify how we execute the callback function to preserve the this object context. Or else the this object will either point to the global window object (in the browser), if callback was passed to a global function. Or it will point to the object of the containing method.
Let’s explore this in code:


// Define an object with some properties and a method
// We will later pass the method as a callback function to another function
var clientData = {
    id: 094545,
    fullName: "Not Set",
    // setUserName is a method on the clientData object
    setUserName: function (firstName, lastName)  {
        // this refers to the fullName property in this object
      this.fullName = firstName + " " + lastName;
    }
}

function getUserInput(firstName, lastName, callback)  {
    // Do other stuff to validate firstName/lastName here

    // Now save the names
    callback (firstName, lastName);
}

In the following code example, when clientData.setUserName is executed, this.fullName will not set the fullName property on the clientData object. Instead, it will set fullName on the window object, since getUserInput is a global function. This happens because the this object in the global function points to the window object.

getUserInput ("Barack", "Obama", clientData.setUserName);

console.log (clientData.fullName);// Not Set

// The fullName property was initialized on the window object
console.log (window.fullName); // Barack Obama

Use the Call or Apply Function To Preserve this
We can fix the preceding problem by using the Call or Apply function (we will discuss these in a full blog post later). For now, know that every function in JavaScript has two methods: Call and Apply. And these methods are used to set the this object inside the function and to pass arguments to the functions.

Call takes the value to be used as the this object inside the function as the first parameter, and the remaining arguments to be passed to the function are passed individually (separated by commas of course). The Apply function’s first parameter is also the value to be used as the this object inside the function, while the last parameter is an array of values (or the arguments object) to pass to the function.

This sounds complex, but lets see how easy it is to use Apply or Call. To fix the problem in the previous example, we will use the Apply function thus:

//Note that we have added an extra parameter for the callback object, called "callbackObj"
function getUserInput(firstName, lastName, callback, callbackObj)  {
    // Do other stuff to validate name here

    // The use of the Apply function below will set the this object to be callbackObj
    callback.apply (callbackObj, [firstName, lastName]);
}

With the Apply function setting the this object correctly, we can now correctly execute the callback and have it set the fullName property correctly on the clientData object:

// We pass the clientData.setUserName method and the clientData object as parameters. The clientData object will be used by the Apply function to set the this object

getUserInput ("Barack", "Obama", clientData.setUserName, clientData);

// the fullName property on the clientData was correctly set
console.log (clientData.fullName); // Barack Obama

We would have also used the Call function, but in this case we used the Apply function.

Multiple Callback Functions Allowed
We can pass more than one callback functions into the parameter of a function, just like we can pass more than one variable. Here is a classic example with jQuery’s AJAX function:

function successCallback() {
    // Do stuff before send
}

function successCallback() {
    // Do stuff if success message received
}

function completeCallback() {
    // Do stuff upon completion
}

function errorCallback() {
    // Do stuff if error received
}

$.ajax({
    url:"http://fiddle.jshell.net/favicon.png",
    success:successCallback,
    complete:completeCallback,
    error:errorCallback

});

“Callback Hell” Problem And Solution

In asynchronous code execution, which is simply execution of code in any order, sometimes it is common to have numerous levels of callback functions to the extent that you have code that looks like the following. The messy code below is called callback hell because of the difficulty of following the code due to the many callbacks. I took this example from the node-mongodb-native, a MongoDB driver for Node.js. [2]. The example code below is just for demonstration:

var p_client = new Db('integration_tests_20', new Server("127.0.0.1", 27017, {}), {'pk':CustomPKFactory});
p_client.open(function(err, p_client) {
    p_client.dropDatabase(function(err, done) {
        p_client.createCollection('test_custom_key', function(err, collection) {
            collection.insert({'a':1}, function(err, docs) {
                collection.find({'_id':new ObjectID("aaaaaaaaaaaa")}, function(err, cursor) {
                    cursor.toArray(function(err, items) {
                        test.assertEquals(1, items.length);

                        // Let's close the db
                        p_client.close();
                    });
                });
            });
        });
    });
});

You are not likely to encounter this problem often in your code, but when you do—and you will from time to time—here are two solutions to this problem. [3]

1. Name your functions and declare them and pass just the name of the function as the callback, instead of defining an anonymous function in the parameter of the main function.
2. Modularity: Separate your code into modules, so you can export a section of code that does a particular job. Then you can import that module into your larger application.

Make Your Own Callback Functions

Now that you completely (I think you do; if not it is a quick reread :)) understand everything about JavaScript callback functions and you have seen that using callback functions are rather simple yet powerful, you should look at your own code for opportunities to use callback functions, for they will allow you to:

• Do not repeat code (DRY—Do Not Repeat Yourself)
• Implement better abstraction where you can have more generic functions that are versatile (can handle all sorts of functionalities)
• Have better maintainability
• Have more readable code
• Have more specialized functions.

It is rather easy to make your own callback functions. In the following example, I could have created one function to do all the work: retrieve the user data, create a generic poem with the data, and greet the user. This would have been a messy function with much if/else statements and, even still, it would have been very limited and incapable of carrying out other functionalities the application might need with the user data.

Instead, I left the implementation for added functionality up to the callback functions, so that the main function that retrieves the user data can perform virtually any task with the user data by simply passing the user’s full name and gender as parameters to the callback function and then executing the callback function.

In short, the getUserInput function is versatile: it can execute all sorts of callback functions with myriad of functionalities.

// First, setup the generic poem creator function; it will be the callback function in the getUserInput function below.
function genericPoemMaker(name, gender) {
    console.log(name + " is finer than fine wine.");
    console.log("Altruistic and noble for the modern time.");
    console.log("Always admirably adorned with the latest style.");
    console.log("A " + gender + " of unfortunate tragedies who still manages a perpetual smile");
}

//The callback, which is the last item in the parameter, will be our genericPoemMaker function we defined above.
function getUserInput(firstName, lastName, gender, callback) {
    var fullName = firstName + " " + lastName;

    // Make sure the callback is a function
    if (typeof callback === "function") {
    // Execute the callback function and pass the parameters to it
    callback(fullName, gender);
    }
}

Call the getUserInput function and pass the genericPoemMaker function as a callback:

getUserInput("Michael", "Fassbender", "Man", genericPoemMaker);
// Output
/* Michael Fassbender is finer than fine wine.
Altruistic and noble for the modern time.
Always admirably adorned with the latest style.
A Man of unfortunate tragedies who still manages a perpetual smile.
*/

Because the getUserInput function is only handling the retrieving of data, we can pass any callback to it. For example, we can pass a greetUser function like this:

function greetUser(customerName, sex)  {
   var salutation  = sex && sex === "Man" ? "Mr." : "Ms.";
  console.log("Hello, " + salutation + " " + customerName);
}

// Pass the greetUser function as a callback to getUserInput
getUserInput("Bill", "Gates", "Man", greetUser);

// And this is the output
Hello, Mr. Bill Gates

We called the same getUserInput function as we did before, but this time it performed a completely different task.

As you see, callback functions afford much versatility. And even though the preceding example is relatively simple, imagine how much work you can save yourself and how well abstracted your code will be if you start using callback functions. Go for it. Do it in the monings; do it in the evenings; do it when you are down; do it when you are k

Note the following ways we frequently use callback functions in JavaScript, especially in modern web application development, in libraries, and in frameworks:


• For asynchronous execution (such as reading files, and making HTTP requests)
• In Event Listeners/Handlers
• In setTimeout and setInterval methods
• For Generalization: code conciseness

Final Words

JavaScript callback functions are wonderful and powerful to use and they provide great benefits to your web applications and code. You should use them when the need arises; look for ways to refactor your code for Abstraction, Maintainability, and Readability with callback functions.

See you next time, and remember to keep coming back because JavaScriptIsSexy.com has much to teach you and you have much to learn.

Notes

1. http://c2.com/cgi/wiki?FirstClass
2. https://github.com/mongodb/node-mongodb-native
3. http://callbackhell.com/
4. JavaScript Patterns by Stoyan Stefanov (Sep 28, 2010)

DecorateThis

DecorateThis (GitHub: mako-taco/DecorateThis, License: Apache 2.0, npm: decorate-this) by Jake Scott is the first library to my knowledge that takes advantage of ES7 decorators. Decorators allow you to annotate classes and properties.

An example of a decorator is @readonly:

class Person {  
  @readonly
  name() { return `${this.first} ${this.last}` }
}

This is far less complex than the older Object.defineProperty syntax:

Object.defineProperty(Person.prototype, 'name', {  
  value: specifiedFunction,
  enumerable: false,
  configurable: true,
  writable: true
});

The DecorateThis module provides some very handy decorators for argument type checking, including:

• Memoization: @memoized
• Type validation: primitive types @param(Number), and a class T is checked with @param(T)
• Duck type checks: @param({ hello: String, info: { age: Number, color: String } });

If these checks fail type errors are thrown, which means you'll need exception handlers for places where these checks should be caught and handled gracefully.

jsRequest

Bruno Panuto and Danilo Valente have been working on a JavaScript file loader called jsRequest (GitHub: danilo-valente/jsRequest, License: MIT) since back in 2013. It handles asynchronous loading, and the syntax supports loading multiple files with a chained API:

jsRequest  
  .load('myScript.js')
  .load('http://code.jquery.com/jquery-1.10.2.js')
  .load('file:///C:/myFile.js');

It supports extra options that allow success and failure callbacks to be supplied, and it also has a progress callback. If you need to order requests, then you can use the wait method.

jsRequest tracks the request history, so if you need to check what files have been loaded you can inspect the jsRequest.history object.

ODataAngularResources

I recently wrote about OData Server, and Raphael Atallah sent in ODataAngularResources (GitHub: devnixs/ODataAngularResources, License: MIT), a fork of Angular's $resource that allows making OData queries using a fluent API.

Once you've added the dependency, you can do cool stuff like this:

var User = $odataresource('/user/:userId', { userId:'@id' });  
var myUsers = User.odata()  
  .filter('Name', 'John')
  .query();

You can also use top, orderBy, and skip, and request single elements. The readme has full examples including details on how to create more complex queries with predicates.

More libraries are starting to blend ideas from functional programming, streams, and reactive programming. Yassine Elouafi sent in ADT Streams (License: MIT, npm: adtstream), a library that offers a stream API using promises and algebraic data types. It has methods for wrapping other streams, like Stream.fromReadable for use with Node's Readable streams, Stream.fromEmitter for objects that extend EventEmitter, and Stream.fromDomEvent for working with DOM events in the browser.

There are utility methods, like Stream.seq for yielding elements from a sequence over time. The blog post, Promises + FP = Beautiful Streams explains each of the library's methods with examples first in Haskell and then in JavaScript. It also has some background on streams and reactive programming:

The basic idea is that you can represent different asynchronous data like events or http responses by an unifying concept called observable collections. Then you can manipulate those collections by standard array operations like map, filter, reduce, forEach ...etc.

By using Yassine's library, you can compose streams with different sources: you can mix events and readable streams with the same API.

The author also wrote From callback to (Future -> Functor -> Monad), which argues why callbacks are not composable and alternative solutions:

With composition, you never hit a wall, because you must always ask yourself "What is the return value of this function?", or put another way "What is the meaning of this thing?"

I don't mind using callbacks in my code. In fact, callbacks are great for things such as representing side effects or event notification, but once your start using them to manage your control flow, you're trapped, why? because they do not compose.

Both of these posts will be of interest to you if you're a JavaScript programmer without much functional or reactive programming experience.

Sometimes I feel like client-side developers are adopting ES6 faster than server-side developers. I certainly have projects where the client uses ES6 and the Node server code is more ES5, mainly because I don't really want to use a transpiler on the server, although I sometimes do like to use Harmony flags.

Alternatively, I could use io.js. Keith Horwood recently sent in a new module that aims to provide an API server and web framework that takes advantage of ES6 features. It's called Nodal (GitHub: keithwhor/nodal, License: MIT, npm: nodal) and currently runs on io.js.

Nodal includes support for models, controllers, templates, migrations, routing, and query composition. It has a command-line tool for creating RESTful resources, and it's designed to work with PostgreSQL.

The example code makes heavy use of new language features like const and classes. The result is very Rails/Django-like. Here's a snippet of a model class:

module.exports = (function() {  
  'use strict';

  const Nodal = require('nodal');

  class Person extends Nodal.Model {
    __preInitialize__() {}
    __postInitialize__() {}
  }

  Person.prototype.schema = Nodal.my.Schema.models.Person;

  Person.prototype.externalInterface = [
    'id',
    'name',
    'age',
    'created_at'
  ];

  return Person;
})();

It doesn't have the same kind of auto-loading magic that you see in Rails -- notice the Nodal module is loaded explicitly. It still feels like idiomatic JavaScript rather than using ES6 to pretend to be something else.

Nodal doesn't use an external ORM, it actually has its own models backed by any-db-postgres. That means Keith is developing features like model relationships. I've tried to develop my own ORM before, and the ORMs I've used before have been very mixed in terms of quality and consistency between releases, so I don't envy the work he has ahead. However, the idea of a RESTful web framework that takes advantage of ES6 for code organisation and clarity is interesting, so let's see what he does with it!

Both ES6 generators and node-fibers can be used to block a coroutine while waiting on some I/O without blocking the entire process.

This means that you can wait for an I/O result within your code but still have the performance benefits of the single-threaded, non-blocking I/O model you all know and love.

Both can do this for arbitrarily deep call stacks. The main difference between the capabilities of the two is how explicit the syntax is. It's the classic battle between wanting to be safe while wanting to be flexible.

---

Generators - Safe, but Annoyingly Explicit

In code that uses generators there is a technique whereby a library will yield continuables or promises and the generator runner will pass in the callback, suspend the generator body, and resume when the callback gets called.

I will use my library, gen-run, in these examples because it's very simple and is the one I know best. There are many others I link to in the gen-run credits.

The Server

We'll start out by writing a simple HTTP server that uses generators to power the request handling logic. It could look something like this.

The Handler

Initially when we write our programs, we often use canned data or otherwise have less I/O to perform to accomplish the given task. In this example, we're using a simple module that returns the result directly.

Now our main app logic can be written. Remember that the server expects us to be in a generator so let's write the logic as a generator.

We want each result to be tagged with the request count as it came into the server. A simple shared variable that gets incremented at the beginning of each request should do the trick.

You can call this with as many concurrent requests as you want. Each will have the correct requestCount because of run-to-finish semantics of JavaScript. Even inside the bodies of generators, arbitrary function calls can't suspend your code.

Modified Handler

Now suppose that down the line, our query function needed to actually do things and perform some I/O.

First we'll change the query function to return a continuable and/or accept a callback. This is a common pattern for many node libraries that need to perform I/O.

We thought ahead and put the body of our handler in a generator to ease the transition, but that function call needs to change. The query function now returns a continuable or expects a callback last. Our handler function can now look like this.

Now we have that big bright yield keyword in there. We can see plainly that our requestCount variable is in danger of a race condition. What happens if a second request comes in while we're still waiting on the query from the first request? They will clobber each other and both requests will be marked as #3, that's what!

Good thing we were forced to change our calling syntax to help us see the danger.

Modified Again

Another transform we could have done is re-write query as a generator itself allowing for deep coroutines with function* at every definition and yield* at every call.

In this case we insert a delegating yield* instead of a plain yield and it works the same as the previous change.

In summary, generators allow for all kinds of nice tricks where you can block on I/O without actually blocking the process, however these require invasive changes to your code (much the same as callbacks act today).

Fibers - Powerful, but Flexible

Long before generators landed in V8 and node.js, Marcel released a library known as node-fibers. This clever little library allows you to use full coroutines in any version of node that the addon compiles against.

I'll repeat the same examples here, but using fibers instead of generators for easy comparison.

The Server

Our server is slightly different because the fiber API is quite different than the gen-run and generator APIs.

We create a fiber for our code to run in. With fibers there is no distinction between generators, iterators, or normal functions. They are all just vanilla functions. Also this means there is no yield or yield*. Only normal function calls.

Also within the body of the fiber, try..catch works just like it does with sync code because the code is blocking within this fiber.

The Handler

We'll start out again with a simple handler. We'll use the same exact query function from before and the same handleRequest except this one is a plain function.

The world is great, everything works and there is no race condition because nobody actually ever suspends the fiber.

The Change

Now again, imagine that some time down the line, the authors of the query function need to perform some I/O to accomplish its task. Well in this case they know they are running inside a fiber. (And even if they didn't know this, they could detect it).

So the only code change required to make query block on I/O is local to the query function.

That's it. The rest of the code will continue to work as before. You test it locally and everything works great. You didn't have to change the signature of any other function that consumed this library.

Then you push this code to production where the server hits a higher concurrency load and people start complaining that their requestCounts aren't always accurate. The problem you discover after many hours of painful debugging is that the query function back in handleRequest that didn't need to be changed syntacticly did change behavior. It suspended the fiber and let other concurrent requests clobber the shared requestCount variable.

Lesson Learned

Next time you complain that adding an async feature to a library causes all consumers of this API to be changed as well, remember that along with that pain is protection. You can't have the protection without the pain.

Both styles of coroutines are powerful with unique strengths and weaknesses.

Safety or flexibility, choose one.