diff --git a/1-js/06-advanced-functions/13-generators/article.md b/1-js/06-advanced-functions/13-generators/article.md index b300e3c7..172ab247 100644 --- a/1-js/06-advanced-functions/13-generators/article.md +++ b/1-js/06-advanced-functions/13-generators/article.md @@ -152,7 +152,7 @@ alert(sequence); // 0, 1, 2, 3 In the code above, `...generateSequence()` turns the iterable into array of items (read more about the spread operator in the chapter [](info:rest-parameters-spread-operator#spread-operator)) -## Converting iterables to generators +## Using generators instead of iterables Some time ago, in the chapter [](info:iterable) we created an iterable `range` object that returns values `from..to`. @@ -203,6 +203,8 @@ alert(sequence); // 1, 2, 3, 4, 5 ...But what if we'd like to keep a custom `range` object? +## Converting Symbol.iterator to generator + We can get the best from both worlds by providing a generator as `Symbol.iterator`: ```js run @@ -220,7 +222,16 @@ let range = { alert( [...range] ); // 1,2,3,4,5 ``` -The `range` object is now iterable. The last variant with a generator is much more concise than the original iterable code, and keeps the same functionality. +The `range` object is now iterable. + +That works pretty well, because when `range[Symbol.iterator]` is called: +- it returns an object (now a generator) +- that has `.next()` method (yep, a generator has it) +- that returns values in the form `{value: ..., done: true/false}` (check, exactly what generator does). + +That's not a coincidence, of course. Generators aim to make iterables easier, so we can see that. + +The last variant with a generator is much more concise than the original iterable code, and keeps the same functionality. ```smart header="Generators may continue forever" In the examples above we generated finite sequences, but we can also make a generator that yields values forever. For instance, an unending sequence of pseudo-random numbers. diff --git a/1-js/09-async/07-async-iteration-generators/article.md b/1-js/09-async/07-async-iteration-generators/article.md index 5042aaa2..8e490273 100644 --- a/1-js/09-async/07-async-iteration-generators/article.md +++ b/1-js/09-async/07-async-iteration-generators/article.md @@ -1,20 +1,297 @@ # Async iteration and generators -In web-programming, we often need to work with fragmented data that comes piece-by-piece. +Asynchronous iterators allow to iterate over data that comes asynchronously, on-demand. -That happens when we upload or download a file. Or we need to fetch paginated data. +For instance, when we download something chunk-by-chunk, or just expect a events to come asynchronously and would like to iterate over that -- async iterators and generators may come in handy. +Let's see a simple example first, to grasp the syntax, and then build a real-life example. -For -Either we need to download or upload something, or we need +## Async iterators -In the previous chapter we saw how `async/await` allows to write asynchronous code. +Asynchronous iterators are totally similar to regular iterators, with a few syntactic differences. -But they don't solve +Here's a small cheatsheet: +| | Iterators | Async iterators | +|-------|-----------|-----------------| +| Object method to provide iteraterable | `Symbol.iterator` | `Symbol.asyncIterator` | +| `next()` return value is | any value | `Promise` | +| to loop, use | `for..of` | `for await..of` | +Let's make an iterable `range` object, like we did in the chapter [](info:iterable), but now it will return values asynchronously, one per second: -Regular iterators work fine with the data that doesn't take time to generate. +```js run +let range = { + from: 1, + to: 5, -A `for..of` loop assumes that we + // for await..of calls this method once in the very beginning +*!* + [Symbol.asyncIterator]() { // (1) +*/!* + // ...it returns the iterator object: + // onward, for await..of works only with that object, asking it for next values + return { + current: this.from, + last: this.to, + + // next() is called on each iteration by the for..of loop +*!* + async next() { // (2) + // it should return the value as an object {done:.., value :...} + // (automatically wrapped into a promise by async) +*/!* + + // can use await inside, do async stuff: + await new Promise(resolve => setTimeout(resolve, 1000)); // (4) + + if (this.current <= this.last) { + return { done: false, value: this.current++ }; + } else { + return { done: true }; + } + } + }; + } +}; + +(async () => { + +*!* + for await (let value of range) { // (3) + alert(value); // 1,2,3,4,5 + } +*/!* + +})() +``` + +As we can see, the components are similar to regular iterators: + +1. To make an object asynchronously iterable, it must have a method `Symbol.asyncIterator` `(1)`. +2. To iterate, we use `for await(let value of range)` `(3)`, namely add "await" after "for". +3. It calls `range[Symbol.asyncIterator]` and expects it to return an object with `next` `(3)` method returning a promise. +4. Then `next()` is called to obtain values. In that example values come with a delay of 1 second `(4)`. + +````warn header="A spread operator doesn't work asynchronously" +Features that require regular, synchronous iterators, don't work with asynchronous ones. + +For instance, a spread operator won't work: +```js +alert( [...range] ); // Error, no Symbol.iterator +``` + +That's natural, as it expects to find `Symbol.iterator`, and there's none. +```` + +## Async generators + +Javascript also provides generators, that are also iterable. + +Let's recall a sequence generator from the chapter [](info:generators): + +```js run +function* generateSequence(start, end) { + for (let i = start; i <= end; i++) { + yield i; + } +} + +for(let value of generateSequence(1, 5)) { + alert(value); // 1, then 2, then 3, then 4, then 5 +} +``` + +Normally, we can't use `await` in generators. But what if we need to? + +No problem, let's make an async generator, like this: + +```js run +*!*async*/!* function* generateSequence(start, end) { + + for (let i = start; i <= end; i++) { + +*!* + // yay, can use await! + await new Promise(resolve => setTimeout(resolve, 1000)); +*/!* + + yield i; + } + +} + +(async () => { + + let generator = generateSequence(1, 5); + for *!*await*/!* (let value of generator) { + alert(value); // 1, then 2, then 3, then 4, then 5 + } + +})(); +``` + +So simple, a little bit magical. We add the `async` keyword, and the generator now can use `await` inside of it, rely on promises and other async functions. + +Technically, the difference of such generator is that `generator.next()` method is now asynchronous. + +Instead of `result = generator.next()` for a regular, non-async generator, values can be obtained like this: + +```js +result = await generator.next(); // result = {value: ..., done: true/false} +``` + +## Iterables via async generators + +When we'd like to make an object iterable, we should add `Symbol.iterator` to it. A common practice is to implement it via a generator, just because that's simple. + +Let's recall an example from the chapter [](info:generators): + +```js run +let range = { + from: 1, + to: 5, + + *[Symbol.iterator]() { // a shorthand for [Symbol.iterator]: function*() + for(let value = this.from; value <= this.to; value++) { + yield value; + } + } +}; + +for(let value of range) { + alert(value); // 1, then 2, then 3, then 4, then 5 +} +``` + +Here a custom object `range` was made iterable, the generator `*[Symbol.iterator]` implements the logic behind listing values. + +We can make it iterable asynchronously by replacing `Symbol.iterator` with `Symbol.asyncIterator` and marking it `async`: + +```js run +let range = { + from: 1, + to: 5, + +*!* + async *[Symbol.asyncIterator]() { // same as [Symbol.asyncIterator]: async function*() +*/!* + for(let value = this.from; value <= this.to; value++) { + + // make a pause between values, wait for something + await new Promise(resolve => setTimeout(resolve, 1000)); + + yield value; + } + } +}; + +(async () => { + + for *!*await*/!* (let value of range) { + alert(value); // 1, then 2, then 3, then 4, then 5 + } + +})(); +``` + +## Real-life example + +There are many online APIs that deliver paginated data. For instance, when we need a list of users, then we can fetch it page-by-page: a request returns a pre-defined count (e.g. 100 users), and provides an URL to the next page. + +The pattern is very common, it's not about users, but just about anything. For instance, Github allows to retrieve commits in the same, paginated fasion: + +- We should make a request to URL in the form `https://api.github.com/repos/(repo)/commits`. +- It responds with a JSON of 30 commits, and also provides a link to the next page in the `Link` header. +- Then we can use it for the next request, if need more commits, and so on. + +What we'd like to have is an iterable source of commits, so that we could do it like this: + +```js +let repo = 'iliakan/javascript-tutorial-en'; // Github repository to get commits from + +for await (let commit of fetchCommits(repo)) { + // process commit +} +``` + +We'd like `fetchCommits` to get commits for us, making requests whenever needed. And let it care about all pagination stuff. + +With async generators that's pretty easy to implement: + +```js +async function* fetchCommits(repo) { + let url = `https://api.github.com/repos/${repo}/commits`; + + while (url) { + const response = await fetch(url, { // (1) + headers: {'User-Agent': 'Our script'}, // github requires user-agent header + }); + + const body = await response.json(); // (2) parses response as JSON (array of commits) + + // (3) the URL of the next page is in the headers, extract it + let nextPage = response.headers.get('Link').match(/<(.*?)>; rel="next"/); + nextPage = nextPage && nextPage[1]; + + url = nextPage; + + for(let commit of body) { // (4) yield commits one by one, until the page ends + yield commit; + } + } +} +``` + +1. We use the browser `fetch` method to download from a remote URL. It allows to supply authorization and other headers if needed (Github requires User-Agent). +2. The result is parsed as JSON. +3. ...And we extract the next page URL from the `Link` header of the response. It has a special format, so we use a regexp for that. The next page URL may look like this: `https://api.github.com/repositories/93253246/commits?page=2`, it's generatd by Github itself. +4. Then we yield all commits received, and when they finish -- the next `while(url)` iteration will trigger, making one more request. + +An example of use (shows commit authors in console): + +```js run +(async () => { + + let count = 0; + + for await (const commit of fetchCommits('iliakan/javascript-tutorial-en')) { + + console.log(commit.author.login); + + if (++count == 100) { // let's stop at 100 commits + break; + } + } + +})(); +``` + +The internal mechanics is invisible from the outside. What we see -- is an async generator that returns commits, just what we need. + +## Summary + +Regular iterators and generators work fine with the data that doesn't take time to generate. + +When we expect the data to come asynchronously, with delays, their async counterparts can be used, and `for await..of` instead of `for..of`. + +Syntax differences between async and regular iterators: + +| | Iterators | Async iterators | +|-------|-----------|-----------------| +| Object method to provide iteraterable | `Symbol.iterator` | `Symbol.asyncIterator` | +| `next()` return value is | any value | `Promise` | + +Syntax differences between async and regular generators: + +| | Generators | Async generators | +|-------|-----------|-----------------| +| Declaration | `function*` | `async function*` | +| `generator.next()` returns | `{value:…, done: true/false}` | `Promise` that resolves to `{value:…, done: true/false}` | + +In web-development we often meet streams of data, when it flows chunk-by-chunk. For instance, downloading or uploading a big file. + +We could use async generators to process such data, but there's also another API called Streams, that may be more convenient. It's not a part of Javascript language standard though. + +Streams and async generators complement each other, both are great ways to handle async data flows. diff --git a/1-js/09-async/07-async-iteration-generators/head.html b/1-js/09-async/07-async-iteration-generators/head.html new file mode 100644 index 00000000..74d66a8b --- /dev/null +++ b/1-js/09-async/07-async-iteration-generators/head.html @@ -0,0 +1,24 @@ +