#CSS #HTML #JavaScript Collection of Code convert tools and utilities.#HTML #CSS #JavaScript … http://pic.twitter.com/H8gqjRWa3v
— Web devel0pment (@devwebbest) September 1, 2016
from Twitter https://twitter.com/devwebbest
Wednesday, August 31, 2016
#CSS #HTML #JavaScript Collection of Code convert tools and utilities. #HTML #CSS #JavaScript … https://t.co/H8gqjRWa3v
from Twitter https://twitter.com/devwebbest
The Portable Guitarist—Hardware
The Portable Guitarist—Hardware
The my first tutorial, The Portable Guitarist—Using iOS as a Live Rig, I explained why an iOS-based live rig has many advantages. In this tutorial, I'll explain the core of the set-up: the hardware.
Before you start, you should consider the reason that you'll be using the device.
Consider Why You'll Be Using an iOS Device
Guitartists often use time-based apps like delays and reverbs. If, however, the device has insufficient memory, the sound will glitch and stutter terribly. Before choosing a device to use, bear this in mind: the more processing power and memory you can get, the better.
Therefore, when looking at the specifications of a preferred device (which you can find on the relevant product page of the Apple website), pay attention to:
- Which generation of processor is onboard, the more recent, the better
- How much memory the device has: be aware that when a 16GB memory is quoted, the reality is nearer 12GB due to system requirements
Don’t spend everything on the device. It may be the centre of your sound set-up, but there’ll be peripherals to buy, so budget accordingly.
For further savings, check out the Refurbished section of the Apple Store. Not only are items cheaper, but they’ll be as-new, and come with a 1-year Apple warranty.
An iPad Is the Sensible Choice
Personally, I favour the iPad. If nothing else, a bigger screen is easier to see on a darkened stage (even if iPhones seem to get bigger with each iteration). Plus, on-screen adjustments are easier; no-one wants to hear you accidentally engaging your death metal tone during a sensitive ballad.
Then there’s battery life; having used both an iPad and iPhone live, the battery life of an iPhone can plummet alarmingly. True, if it’s fully charged to start with, you’re unlikely to play long enough to drain it down, and a break between sets lets you recharge it. However, I’d rather focus on my performance than the battery icon.
Remember to check the Apple store for refurbished iPads.
Don't Dismiss the iPhone or iPod Touch
Screen size and battery life aside, the iPhone and iPod Touch suffer little in specification; currently, the iPhone 6, 6 Plus, and iPod Touch all share the same processor as the iPad mini 4, and are all available up to 64GB. The iPhone 6S and 6S Plus offer improved processor and a maximum of 128GB memory.
It can also be cheaper; Apple currently charges £159 for the 16GB iPod Touch. If you can afford it, the top-of-the-range 128GB is £329; compare this to the iPhone 6S at £539, and the iPad Air 2 at £559.
So they can be cheaper, and they’re certainly even more portable. They generally run the same apps as the iPad, certainly the guitar-centric ones, and accept the same interfaces.
Interfaces
There are generally two types of interface:
- headphone socket, and
- dock
Headphone Socket
These were the original iOS interface, and the iRig by IK Multimedia is probably the most famous. Beautifully straightforward, and attractively cheap, it was a trailblazer.
In 2015 it was superseded by the iRig2, which is a little bigger, and has more features such as input gain control. Crucially, it’s still cheap; the iRig2 costs under £30. The original iRig is still available, and I’ve found it for as little as £15 new.
However, this wonderful cheapness comes at a price: noise. Lots and lots of noise.
The headphone socket is supposed to be for output, so introducing an input signal in such close proximity creates hiss, and can lead to feedback.
The Ampkit Link from Peavey tried to address this by being battery-powered, and it certainly meant a higher signal level could be achieved before feedback occurred. But it didn’t cure it, so guitarists relied heavily on noise gates.
A better solution is that of a dock interface.
Dock Interfaces
Unlike the headphone socket, the dock doesn’t rely on just three connections, so input and output signals can be kept separate, significantly cleaning up the sound. It also frees the headphone socket to do what it was designed to do.
Of the models on the market, and ignoring those of a desktop nature, my choice is the Jam from Apogee. It’s extremely compact, and requires no external power source.
Other features include the integral gain control which means it can also be used with certain microphones. It is compatible with both iOS and Mac for live performance and home recording. It also still serves the 30-pin format of older devices, alongside that of current, Lightning-equipped ones.
Quality of sound is what matters, and the Jam’s 24-bit, 48kHz digital converters don’t disappoint. If that isn’t good enough for you, there’s a 96kHz version.
All of this sonic goodness does come at a higher price, however; a 48kHz Jam is typically over £70, and the 96Khz is over £100. If you want to go slightly cheaper, check out the SonicPort from Line 6.
Hands Free
Using iOS live has meant having one's hands more on a screen than perhaps many guitarists would like; this becomes increasingly applicable if you use lots of different sounds.
Bluetooth-based foot controllers, however, exploit the fact that many apps accept MIDI commands. Consequently, switching pedals onscreen becomes no different to that of kicking old-school stompboxes on and off.
As for what’s available, the iRig BlueBoard from IK Multimedia has lots to recommend it; four backlit footpads, expression pedal inputs, fits in a gig bag, and works with most iDevices, as well as Mac.
If plastic seems flimsy, check out the all-metal BT-4 from Positive Grid; more expensive, but more robust.
Conclusion
In looking at the core of a live set-up, I have shown you that
- An iPad is a preferable option
- An iPhone or iPod Touch are be cheaper and aren’t necessarily a poorer choice
- Go for the highest specification you can afford
- Apple Refurbished models can save you money
- A dock interface is the only serious choice for sound quality
- A Bluetooth foot controller brings familiar analogue functionality
In the next tutorial, I’ll show you how to mount and connect equipment effectively for a live environment.
Let's Go: Command-Line Programs With Golang
Let's Go: Command-Line Programs With Golang
Overview
The Go Language is an exciting new language that gains a lot of popularity for a good reason. In this tutorial you'll learn how to write command-line programs with Go. The sample program is called multi-git, and it allows you to execute git commands on multiple repositories at the same time.
Quick Introduction to Go
Go is an open-source C-like language created at Google by some of the original C and Unix hackers, who were motivated by their dislike of C++. It shows in Go's design, which made several unorthodox choices such as eschewing implementation inheritance, templates, and exceptions. Go is simple, reliable, and efficient. Its most distinctive feature is its explicit support for concurrent programming via so-called goroutines and channels.
Before starting to dissect the sample program, follow the official guide to get ready for Go development.
The Multi-Git Program
The multi-git program is a simple but useful Go program. If you work on a team where the codebase is split across multiple git repositories then you often need to perform changes across multiple repositories. This is a problem because git has no concept of multiple repositories. Everything revolves around a single repository.
This becomes especially troublesome if you use branches. If you work on a feature that touches three repositories then you will have to create a feature branch in each of these repositories and then remember to check out, pull, push, and merge all of them at the same time. This is not trivial. Multi-git manages a set of repositories and lets you operate on the whole set at once. Note that the current version of multi-git requires that you create the branches individually, but I may add this feature at a later date.
By exploring the way multi-git is implemented, you will learn a lot about writing command-line programs in Go.
Packages and Imports
Go programs are organized in packages. The multi-git program consists of a single file called main.go. At the top of the file, the package name 'main' is specified, followed by a list of imports. The imports are other packages that are used by multi-git.
package main
import (
"flag"
"fmt"
"log"
"os"
"strings"
"os/exec"
)
For example, the fmt package is used for formatted I/O similar to C's printf and scanf. Go supports installing packages from a variety of sources via the go get command. When you install packages, they end up in a namespace under the $GOPATH environment variable. You can install packages from a variety of sources such as GitHub, Bitbucket, Google code, Launchpad, and even IBM DevOps services via several common version control formats such as git, subversion, mercurial and bazaar.
Command-Line Arguments
Command-line arguments are one of the most common forms of providing input to programs. They are easy to use, allow you to run and configure the program in one line, and have great parsing support in many languages. Go calls them command-line "flags" and has the flag package for specifying and parsing command-line arguments (or flags).
Typically, you parse command-line arguments at the beginning of your program, and multi-git follows this convention. The entry point is the main() function. The first two lines define two flags called "command" and "ignoreErrors". Each flag has a name, a data type, a default value, and a help string. The flag.Parse() call will parse the actual command-line passed to the program and will populate the defined flags.
func main() {
command := flag.String("command", "", "The git command")
ignoreErrors := flag.Bool(
"ignore-errors",
false,
"Keep running after error if true")
flag.Parse()
It is also possible to access undefined arguments via the flag.Args() function. So, flags stand for pre-defined arguments and "args" are unprocessed arguments. The unprocessed arguments are 0-based indexed.
Environment Variables
Another common form of program configuration is environment variables. When you use environment variables, you may run the same program multiple times in the same environment, and all runs will use the same environment variables.
Multi-git uses two environment variables: "MG_ROOT" and "MG_REPOS". Multi-git is designed to manage a group of git repositories that have a common parent directory. That's "MG_ROOT". The repository names are specified in "MG_REPOS" as a comma-separated string. To read the value of an environment variable you can use the os.Getenv() function.
// Get managed repos from environment variables
root := os.Getenv("MG_ROOT")
if root[len(root) - 1] != '/' {
root += "/"
}
repo_names := strings.Split(os.Getenv("MG_REPOS"), ",")
Verifying the Repository List
Now that it found the root directory and the names of all the repositories, multi-git verifies that each repository exists under root and that it is really a git repository. The check is as simple as looking for a .git sub-directory for each repository directory.
First, an array of strings named "repos" is defined. Then it iterates over all the repo names and constructs a repository path by concatenating the root directory and the repo name. If the [os.Stat()]() call fails for the .git subdirectory, it logs the error and exits. Otherwise, the repository path is appended to the repos array.
var repos []string
// Verify all repos exist and are actually git repos (have .git sub-dir)
for _, r := range repo_names {
path := root + r
_, err := os.Stat(path + "/.git")
if err != nil {
log.Fatal(err)
}
repos = append(repos, path)
}
Go has a unique error-handling facility where functions often return both a return value and an error object. Check out how os.Stat() returns two values. In this case the "_" placeholder is used to hold the actual result because you only care about the error. Go is very strict and requires named variables to be used. If you don't plan to use a value, you should assign it to "_" to avoid compilation error.
Executing Shell Commands
At this point, you have your list of repository paths where we want to execute the git command. As you recall, we received the git command line as a single command-line argument (flag) called "command". This needs to be split into an array of components (git command, sub-command, and options). The whole command as a string is stored too for display purposes.
// Break the git command into components (needed to execute)
var git_components []string
for _, component := range strings.Split(*command, " ") {
git_components = append(git_components, component)
}
command_string := "git " + *command
Now, you're all set to iterate over each repository and execute the git command in each one. The "for ... range" loop construct is used again. First, multi-git changes its working directory to the current target repo "r" and prints the git command. Then it executes the command using the exec.Command() function and prints the combined output (both standard output and standard error).
Finally, it checks if there was an error during execution. If there was an error and the ignoreErrors flag is false then multi-git bails out. The reason for optionally ignoring errors is that sometimes it's OK if commands fail on some repos. For example, if you want to check out a branch called "cool feature" on all the repositories that have this branch, you don't care if the checkout fails on repositories that don't have this branch.
for _, r := range repos {
// Go to the repo's directory
os.Chdir(r);
// Print the command
fmt.Printf("[%s] %s\n", r, command_string)
// Execute the command
out, err := exec.Command("git", git_components...).CombinedOutput()
// Print the result
fmt.Println(string(out))
// Bail out if there was an error and NOT ignoring errors
if err != nil && !*ignoreErrors {
os.Exit(1)
}
}
fmt.Println("Done.")
Conclusion
Go is a simple yet powerful language. It's designed for large-scale system programming but works just fine for small command-line programs too. Go's minimal design is in stark contrast to other modern languages like Scale and Rust that are very powerful and well-designed too, but have a very steep learning curve. I encourage you to try Go and experiment. It's a lot of fun.
Tuesday, August 30, 2016
Monday, August 29, 2016
How to Stabilize Video in After Effects with ReelSteady
How to Stabilize Video in After Effects with ReelSteady
In this tutorial you will learn how to get started using the plug-in ReelSteady to stabilize footage in Adobe After Effects. We will take a look at a few common scenarios shot on various formats, such as a DSLR and GoPro cameras. You will also learn about the best settings to use on your camera before filming shots you plan on stabilizing in post.
What You Need
Besides After Effects, in order to follow along with this lesson you will need to download the plug-in ReelSteady. A free-trial version is available to download as well, so you can easily follow along with this tutorial and experiment using ReelSteady on your own footage.
Tips before you Shoot
- Shoot with a high shutter speed (I usually shoot around 200 to 320.)
- Use a wide-angle lens. A wide-angle or fish-eye lens work best, but if you don't have one of those, just shoot with the widest lens you have available.
- Shoot in 4K, or the highest resolution you can. More pixels means more tracking data and less of a chance for pixelization to occur.
- Film using the 'Tripod Trick'. If you have a tripod, attached it to the bottom of your camera and hold it while moving or walking. This will help prevent any high frequency vibration that can cause excessive rolling shutter artifacts.
- Be aware of parallax between objects in your scene.
How to Use ReelSteady to Stabilize Footage in Adobe After Effects
Links Mentioned
Serialization and Deserialization of Python Objects: Part 2
Serialization and Deserialization of Python Objects: Part 2
This is part two of a tutorial on serializing and deserializing Python objects. In part one, you learned the basics and then dove into the ins and outs of Pickle and JSON.
In this part you'll explore YAML (make sure to have the running example from part one), discuss performance and security considerations, get a review of additional serialization formats, and finally learn how to choose the right scheme.
YAML
YAML is my favorite format. It is a human-friendly data serialization format. Unlike Pickle and JSON, it is not part of the Python standard library, so you need to install it:
pip install yaml
The yaml module has only load() and dump() functions. By default they work with strings like loads() and dumps(), but can take a second argument, which is an open stream and then can dump/load to/from files.
import yaml print yaml.dump(simple) boolean: true int_list: [1, 2, 3] none: null number: 3.44 text: string
Note how readable YAML is compared to Pickle or even JSON. And now for the coolest part about YAML: it understands Python objects! No need for custom encoders and decoders. Here is the complex serialization/deserialization using YAML:
> serialized = yaml.dump(complex)
> print serialized
a: !!python/object:__main__.A
simple:
boolean: true
int_list: [1, 2, 3]
none: null
number: 3.44
text: string
when: 2016-03-07 00:00:00
> deserialized = yaml.load(serialized)
> deserialized == complex
True
As you can see, YAML has its own notation to tag Python objects. The output is still very human readable. The datetime object doesn't require any special tagging because YAML inherently supports datetime objects.
Performance
Before you start thinking of performance, you need to think if performance is a concern at all. If you serialize/deserialize a small amount of data relatively infrequently (e.g. reading a config file at the beginning of a program) then performance is not really a concern and you can move on.
But, assuming you profiled your system and discovered that serialization and/or deserialization are causing performance issues, here are the things to address.
The are two aspects for performance: how fast can you serialize/deserialize, and how big is the serialized representation?
To test the performance of the various serialization formats, I'll create a largish data structure and serialize/deserialize it using Pickle, YAML, and JSON. The big_data list contains 5,000 complex objects.
big_data = [dict(a=simple, when=datetime.now().replace(microsecond=0)) for i in range(5000)]
Pickle
I'll use IPython here for its convenient %timeit magic function that measures execution times.
import cPickle as pickle In [190]: %timeit serialized = pickle.dumps(big_data) 10 loops, best of 3: 51 ms per loop In [191]: %timeit deserialized = pickle.loads(serialized) 10 loops, best of 3: 24.2 ms per loop In [192]: deserialized == big_data Out[192]: True In [193]: len(serialized) Out[193]: 747328
The default pickle takes 83.1 milliseconds to serialize and 29.2 milliseconds to deserialize, and the serialized size is 747,328 bytes.
Let's try with the highest protocol.
In [195]: %timeit serialized = pickle.dumps(big_data, protocol=pickle.HIGHEST_PROTOCOL) 10 loops, best of 3: 21.2 ms per loop In [196]: %timeit deserialized = pickle.loads(serialized) 10 loops, best of 3: 25.2 ms per loop In [197]: len(serialized) Out[197]: 394350
Interesting results. The serialization time shrank to only 21.2 milliseconds, but the deserialization time increased a little to 25.2 milliseconds. The serialized size shrank significantly to 394,350 bytes (52%).
JSON
In [253] %timeit serialized = json.dumps(big_data, cls=CustomEncoder) 10 loops, best of 3: 34.7 ms per loop In [253] %timeit deserialized = json.loads(serialized, object_hook=decode_object) 10 loops, best of 3: 148 ms per loop In [255]: len(serialized) Out[255]: 730000
Ok. Performance seems to be a little worse than Pickle for encoding, but much, much worse for decoding: 6 times slower. What's going on? This is an artifact of the object_hook function that needs to run for every dictionary to check if it needs to convert it to an object. Running without the object hook is much faster.
%timeit deserialized = json.loads(serialized) 10 loops, best of 3: 36.2 ms per loop
The lesson here is that when serializing and deserializing to JSON, consider very carefully any custom encodings because they may have a major impact on the overall performance.
YAML
In [293]: %timeit serialized = yaml.dump(big_data) 1 loops, best of 3: 1.22 s per loop In[294]: %timeit deserialized = yaml.load(serialized) 1 loops, best of 3: 2.03 s per loop In [295]: len(serialized) Out[295]: 200091
Ok. YAML is really, really slow. But, note something interesting: the serialized size is just 200,091 bytes. Much better than both Pickle and JSON. Let's look inside real quick:
In [300]: print serialized[:211]
- a: &id001
boolean: true
int_list: [1, 2, 3]
none: null
number: 3.44
text: string
when: 2016-03-13 00:11:44
- a: *id001
when: 2016-03-13 00:11:44
- a: *id001
when: 2016-03-13 00:11:44
YAML is being very clever here. It identified that all 5,000 dicts share the same value for the 'a' key, so it stores it only once and references it using *id001 for all objects.
Security
Security is an often a critical concern. Pickle and YAML, by virtue of constructing Python objects, are vulnerable to code execution attacks. A cleverly formatted file can contain arbitrary code that will be executed by Pickle or YAML. There is no need to be alarmed. This is by design and is documented in Pickle's documentation:
Warning: The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.
As well as in YAML's documentation:
Warning: It is not safe to call yaml.load with any data received from an untrusted source! yaml.load is as powerful as pickle.load and so may call any Python function.
You just need to understand that you shouldn't load serialized data received from untrusted sources using Pickle or YAML. JSON is OK, but again if you have custom encoders/decoders than you may be exposed, too.
The yaml module provides the yaml.safe_load() function that will load only simple objects, but then you lose a lot of YAML's power and maybe opt to just use JSON.
Other Formats
There are many other serialization formats available. Here are a few of them.
Protobuf
Protobuf, or protocol buffers, is Google's data interchange format. It is implemented in C++ but has Python bindings. It has a sophisticated schema and packs data efficiently. Very powerful, but not very easy to use.
MessagePack
MessagePack is another popular serialization format. It is also binary and efficient, but unlike Protobuf it doesn't require a schema. It has a type system that's similar to JSON, but a little richer. Keys can be any type, and not just strings and non-UTF8 strings are supported.
CBOR
CBOR stands for Concise Binary Object Representation. Again, it supports the JSON data model. CBOR is not as well-known as Protobuf or MessagePack but is interesting for two reasons:
- It is an official Internet standard: RFC 7049.
- It was designed specifically for the Internet of Things (IoT).
How to Choose?
This is the big question. With so many options, how do you choose? Let's consider the various factors that should be taken into account:
- Should the serialized format be human-readable and/or human-editable?
- Is serialized content going to be received from untrusted sources?
- Is serialization/deserialization a performance bottleneck?
- Does serialized data need to be exchanged with non-Python environments?
I'll make it very easy for you and cover several common scenarios and which format I recommend for each one:
Auto-Saving Local State of a Python Program
Use pickle (cPickle) here with the HIGHEST_PROTOCOL. It's fast, efficient and can store and load most Python objects without any special code. It can be used as a local persistent cache also.
Configuration Files
Definitely YAML. Nothing beats its simplicity for anything humans need to read or edit. It's used successfully by Ansible and many other projects. In some situations, you may prefer to use straight Python modules as configuration files. This may be the right choice, but then it's not serialization, and it's really part of the program and not a separate configuration file.
Web APIs
JSON is the clear winner here. These days, Web APIs are consumed most often by JavaScript web applications that speak JSON natively. Some Web APIs may return other formats (e.g. csv for dense tabular result sets), but I would argue that you can package csv data into JSON with minimal overhead (no need to repeat each row as an object with all the column names).
High-Volume / Low-Latency Large-Scale Communication
Use one of the binary protocols: Protobuf (if you need a schema), MessagePack, or CBOR. Run your own tests to verify the performance and the representative power of each option.
Conclusion
Serialization and deserialization of Python objects is an important aspect of distributed systems. You can't send Python objects directly over the wire. You often need to interoperate with other systems implemented in other languages, and sometimes you just want to store the state of your program in persistent storage.
Python comes with several serialization schemes in its standard library, and many more are available as third-party modules. Being aware of all the options and the pros and cons of each one will let you choose the best method for your situation.
Sunday, August 28, 2016
Friday, August 26, 2016
Thursday, August 25, 2016
Serialization and Deserialization of Python Objects: Part 1
Serialization and Deserialization of Python Objects: Part 1
Python object serialization and deserialization is an important aspect of any non-trivial program. If in Python you save something to a file, if you read a configuration file, or if you respond to an HTTP request, you do object serialization and deserialization.
In one sense, serialization and deserialization are the most boring things in the world. Who cares about all the formats and protocols? You just want to persist or stream some Python objects and get them back later intact.
This is a very healthy way to look at the world at the conceptual level. But, at the pragmatic level, which serialization scheme, format or protocol you choose may determine how fast your program runs, how secure it is, how much freedom you have to maintain your state, and how well you're going to interoperate with other systems.
The reason there are so many options is that different circumstances call for different solutions. There is no "one size fits all". In this two-part tutorial I'll go over the pros and cons of the most successful serialization and deserialization schemes, show how to use them, and provide guidelines for choosing between them when faced with a specific use case.
Running Example
In the following sections I'll serialize and deserialize the same Python object graphs using different serializers. To avoid repetition, I'll define these object graphs here.
Simple Object Graph
The simple object graph is a dictionary that contains a list of integers, a string, a float, a boolean, and a None.
simple = dict(int_list=[1, 2, 3],
text='string',
number=3.44,
boolean=True,
none=None)
Complex Object Graph
The complex object graph is also a dictionary, but it contains a datetime object and user-defined class instance that has a self.simple attribute, which is set to the simple object graph.
from datetime import datetime
class A(object):
def __init__(self, simple):
self.simple = simple
def __eq__(self, other):
if not hasattr(other, 'simple'):
return False
return self.simple == other.simple
def __ne__(self, other):
if not hasattr(other, 'simple'):
return True
return self.simple != other.simple
complex = dict(a=A(simple), when=datetime(2016, 3, 7))
Pickle
Pickle is a staple. It is a native Python object serialization format. The pickle interface provides four methods: dump, dumps, load, and loads. The dump() method serializes to an open file (file-like object). The dumps() method serializes to a string. The load() method deserializes from an open file-like object. The loads() method deserializes from a string.
Pickle supports by default a textual protocol, but has also a binary protocol, which is more efficient, but not human-readable (helpful when debugging).
Here is how you pickle a Python object graph to a string and to a file using both protocols.
import cPickle as pickle pickle.dumps(simple) "(dp1\nS'text'\np2\nS'string'\np3\nsS'none'\np4\nNsS'boolean'\np5\nI01\nsS'number'\np6\nF3.4399999999999999\nsS'int_list'\np7\n(lp8\nI1\naI2\naI3\nas." pickle.dumps(simple, protocol=pickle.HIGHEST_PROTOCOL) '\x80\x02}q\x01(U\x04textq\x02U\x06stringq\x03U\x04noneq\x04NU\x07boolean\x88U\x06numberq\x05G@\x0b\x85\x1e\xb8Q\xeb\x85U\x08int_list]q\x06(K\x01K\x02K\x03eu.'
The binary representation may seem larger, but this is an illusion due to its presentation. When dumping to a file, the textual protocol is 130 bytes, while the binary protocol is only 85 bytes.
pickle.dump(simple, open('simple1.pkl', 'w'))
pickle.dump(simple, open('simple2.pkl', 'wb'), protocol=pickle.HIGHEST_PROTOCOL)
ls -la sim*.*
-rw-r--r-- 1 gigi staff 130 Mar 9 02:42 simple1.pkl
-rw-r--r-- 1 gigi staff 85 Mar 9 02:43 simple2.pkl
Unpickling from a string is as simple as:
x = pickle.loads("(dp1\nS'text'\np2\nS'string'\np3\nsS'none'\np4\nNsS'boolean'\np5\nI01\nsS'number'\np6\nF3.4399999999999999\nsS'int_list'\np7\n(lp8\nI1\naI2\naI3\nas.")
assert x == simple
x = pickle.loads('\x80\x02}q\x01(U\x04textq\x02U\x06stringq\x03U\x04noneq\x04NU\x07boolean\x88U\x06numberq\x05G@\x0b\x85\x1e\xb8Q\xeb\x85U\x08int_list]q\x06(K\x01K\x02K\x03eu.')
assert x == simple
Note that pickle can figure out the protocol automatically. There is no need to specify a protocol even for the binary one.
Unpickling from a file is just as easy. You just need to provide an open file.
x = pickle.load(open('simple1.pkl'))
assert x == simple
x = pickle.load(open('simple2.pkl'))
assert x == simple
x = pickle.load(open('simple2.pkl', 'rb'))
assert x == simple
According to the documentation, you're supposed to open binary pickles using the 'rb' mode, but as you can see it works either way.
Let's see how pickle deals with the complex object graph.
pickle.dumps(complex)
"(dp1\nS'a'\nccopy_reg\n_reconstructor\np2\n(c__main__\nA\np3\nc__builtin__\nobject\np4\nNtRp5\n(dp6\nS'simple'\np7\n(dp8\nS'text'\np9\nS'string'\np10\nsS'none'\np11\nNsS'boolean'\np12\nI01\nsS'number'\np13\nF3.4399999999999999\nsS'int_list'\np14\n(lp15\nI1\naI2\naI3\nassbsS'when'\np16\ncdatetime\ndatetime\np17\n(S'\\x07\\xe0\\x03\\x07\\x00\\x00\\x00\\x00\\x00\\x00'\ntRp18\ns."
pickle.dumps(complex, protocol=pickle.HIGHEST_PROTOCOL)
'\x80\x02}q\x01(U\x01ac__main__\nA\nq\x02)\x81q\x03}q\x04U\x06simpleq\x05}q\x06(U\x04textq\x07U\x06stringq\x08U\x04noneq\tNU\x07boolean\x88U\x06numberq\nG@\x0b\x85\x1e\xb8Q\xeb\x85U\x08int_list]q\x0b(K\x01K\x02K\x03eusbU\x04whenq\x0ccdatetime\ndatetime\nq\rU\n\x07\xe0\x03\x07\x00\x00\x00\x00\x00\x00\x85Rq\x0eu.'
pickle.dump(complex, open('complex1.pkl', 'w'))
pickle.dump(complex, open('complex2.pkl', 'wb'), protocol=pickle.HIGHEST_PROTOCOL)
ls -la comp*.*
-rw-r--r-- 1 gigi staff 327 Mar 9 02:58 complex1.pkl
-rw-r--r-- 1 gigi staff 171 Mar 9 02:58 complex2.pkl
The efficiency of the binary protocol is even greater with complex object graphs.
JSON
JSON (JavaScript Object Notation) has been part of the Python standard library since Python 2.5. I'll consider it a native format at this point. It is a text-based format and is the unofficial king of the web as far as object serialization goes. Its type system naturally models JavaScript, so it is pretty limited.
Let's serialize and deserialize the simple and complex objects graphs and see what happens. The interface is almost identical to the pickle interface. You have dump(), dumps(), load(), and loads() functions. But, there are no protocols to select, and there are many optional arguments to control the process. Let's start simple by dumping the simple object graph without any special arguments:
import json
print json.dumps(simple)
{"text": "string", "none": null, "boolean": true, "number": 3.44, "int_list": [1, 2, 3]}
The output looks pretty readable, but there is no indentation. For a larger object graph, this can be a problem. Let's indent the output:
print json.dumps(simple, indent=4)
{
"text": "string",
"none": null,
"boolean": true,
"number": 3.44,
"int_list": [
1,
2,
3
]
}
That looks much better. Let's move on to the complex object graph.
json.dumps(complex)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-19-1be2d89d5d0d> in <module>()
----> 1 json.dumps(complex)
/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.pyc in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, encoding, default, sort_keys, **kw)
241 cls is None and indent is None and separators is None and
242 encoding == 'utf-8' and default is None and not sort_keys and not kw):
--> 243 return _default_encoder.encode(obj)
244 if cls is None:
245 cls = JSONEncoder
/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.pyc in encode(self, o)
205 # exceptions aren't as detailed. The list call should be roughly
206 # equivalent to the PySequence_Fast that ''.join() would do.
--> 207 chunks = self.iterencode(o, _one_shot=True)
208 if not isinstance(chunks, (list, tuple)):
209 chunks = list(chunks)
/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.pyc in iterencode(self, o, _one_shot)
268 self.key_separator, self.item_separator, self.sort_keys,
269 self.skipkeys, _one_shot)
--> 270 return _iterencode(o, 0)
271
272 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.pyc in default(self, o)
182
183 """
--> 184 raise TypeError(repr(o) + " is not JSON serializable")
185
186 def encode(self, o):
TypeError: <__main__.A object at 0x10f367cd0> is not JSON serializable
Whoa! That doesn't look good at all. What happened? The error message is that the A object is not JSON serializable. Remember that JSON has a very limited type system and it can't serialize user defined classes automatically. The way to address it is to subclass the JSONEncoder class used by the json module and implement the default() that is called whenever the JSON encoder runs into an object it can't serialize.
The job of the custom encoder is to convert it to a Python object graph that the JSON encoder is able to encode. In this case we have two objects that require special encoding: the datetime object and the A class. The following encoder does the job. Each special object is converted to a dict where the key is the name of the type surrounded by dunders (double underscores). This will be important for decoding.
from datetime import datetime
import json
class CustomEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, datetime):
return {'__datetime__': o.replace(microsecond=0).isoformat()}
return {'__{}__'.format(o.__class__.__name__): o.__dict__}
Let's try again with our custom encoder:
serialized = json.dumps(complex, indent=4, cls=CustomEncoder)
print serialized
{
"a": {
"__A__": {
"simple": {
"text": "string",
"none": null,
"boolean": true,
"number": 3.44,
"int_list": [
1,
2,
3
]
}
}
},
"when": {
"__datetime__": "2016-03-07T00:00:00"
}
}
This is beautiful. The complex object graph was serialized properly, and the original type information of the components was retained via the keys: "__A__" and "__datetime__". If you use dunders for your names, then you need to come up with a different convention to denote special types.
Let's decode the complex object graph.
> deserialized = json.loads(serialized) > deserialized == complex False
Hmmm, the deserialization worked (no errors), but it is different than the original complex object graph we serialized. Something is wrong. Let's take a look at the deserialized object graph. I'll use the pprint function of the pprint module for pretty printing.
> from pprint import pprint
> pprint(deserialized)
{u'a': {u'__A__': {u'simple': {u'boolean': True,
u'int_list': [1, 2, 3],
u'none': None,
u'number': 3.44,
u'text': u'string'}}},
u'when': {u'__datetime__': u'2016-03-07T00:00:00'}}
Ok. The problem is that the json module doesn't know anything about the A class or even the standard datetime object. It just deserializes everything by default to the Python object that matches its type system. In order to get back to a rich Python object graph, you need custom decoding.
There is no need for a custom decoder subclass. The load() and loads() functions provide the "object_hook" parameter that lets you provide a custom function that converts dicts to objects.
def decode_object(o):
if '__A__' in o:
a = A()
a.__dict__.update(o['__A__'])
return a
elif '__datetime__' in o:
return datetime.strptime(o['__datetime__'], '%Y-%m-%dT%H:%M:%S')
return o
Let's decode using the decode_object() function as a parameter to the loads() object_hook parameter.
> deserialized = json.loads(serialized, object_hook=decode_object)
> print deserialized
{u'a': <__main__.A object at 0x10d984790>, u'when': datetime.datetime(2016, 3, 7, 0, 0)}
> deserialized == complex
True
Conclusion
In part one of this tutorial, you've learned about the general concept of serialization and deserialization of Python objects and explored the ins and out of serializing Python objects using Pickle and JSON.
In part two, you'll learn about YAML, performance and security concerns, and a quick review of additional serialization schemes.
Wednesday, August 24, 2016
Tuesday, August 23, 2016
Monday, August 22, 2016
Bridging React With Other Popular Web Languages
Bridging React With Other Popular Web Languages
React is a view library written in JavaScript, and so it is agnostic of any stack configuration and can make an appearance in practically any web application that is using HTML and JavaScript for its presentation layer.
As React works as the ‘V’ in ‘MVC’, we can create our own application stack from our preferences. So far in this guide we have seen React working with Express, a Node ES6/JavaScript-based framework. Other popular Node-based matches for React are the Meteor framework and Facebook’s Relay.
If you want to take advantage of React’s excellent component-based JSX system, the virtual DOM and its super-fast rendering times with your existing project, you can do so by implementing one of the many open-source solutions.
PHP
As PHP is a server-side scripting language, integration with React can come in several forms:
- Using react-php-v8js
- Server-side Request/Response Route handling (with a router, for example Alto)
- JSON output via json_encode()
- Template wrapping, e.g. Twig
Server-Side Rendering
For rendering React components on the server, there is a library available on GitHub.
For example, we can do the following in PHP with this package:
<?php
// the library
$react_source = file_get_contents('/path/to/build/react.js');
// all custom code concatenated
$app_source = file_get_contents('/path/to/custom/components.js');
$rjs = new ReactJS($react_source, $app_source);
$rjs->setComponent('MyComponent', array(
'any' => 1,
'props' => 2
)
);
// print rendered markup
echo '<div id="here">' . $rjs->getMarkup() . '</div>';
// load JavaScript somehow - concatenated, from CDN, etc
// including react.js and custom/components.js
// init client
echo '<script>' . $rjs->getJS("#here") . '</script>';
// repeat setComponent(), getMarkup(), getJS() as necessary
// to render more components
The power of combining React with any server-side scripting language is the ability to feed React with data, and apply your business logic on the server as well as the client side. Renovating an old application into a Reactive app has never been easier!
Using PHP + Alto Router
For an example application, take a look at this repository on GitHub.
Configure your AltoRouter as so:
<?php
// Router setup
require_once 'include/AltoRouter/AltoRouter.php';
$router = new AltoRouter();
$viewPath = 'views/';
// Router matches
//---
// Manual
$router->map('GET', '/', $viewPath . 'reactjs.html', 'reactjs');
$result = $viewPath . '404.php';
$match = $router->match();
if($match) {
$result = $match['target'];
}
// Return route match
include $result;
?>
With the AltoRouter setup serving your application’s pages for the routes specified, you can then just include your React code inside the HTML markup and begin using your components.
JavaScript:
"use strict";
var Comment = React.createClass({
displayName: "Comment",
render: function render() {
var rawMarkup = marked(this.props.children.toString(), { sanitize: true });
return React.createElement(
"div",
{ className: "comment" },
React.createElement(
"h2",
{ className: "commentAuthor" },
this.props.author
),
React.createElement("span", { dangerouslySetInnerHTML: { __html: rawMarkup } })
);
}
});
Ensure you include the React libraries and also place the HTML inside the body tag that will be served from your PHP AltoRouter app, for example:
<!DOCTYPE html>
<html>
<head>
<title>React Example</title>
<script src="http://ift.tt/1IHLqnE"></script>
<script src="http://ift.tt/1btuHal"></script>
<script src="http://ift.tt/1jyb1yI"></script>
<script src="http://ift.tt/1JmkwQq"></script>
</head>
<body>
<div id="myContent"></div>
<script type="text/jsx;harmony=true" src="assets/js/main.js"></script>
</body>
</html>
Laravel Users
For the highly popular PHP framework Laravel, there is the react-laravel library, which enables React.js from right inside your Blade views.
For example:
@react_component('Message', [ 'title' => 'Hello, World' ], [ 'prerender' => true ])
The prerender flag tells Laravel to render the component on the server side and then mount it to the client side.
Example Laravel 5.2 + React App
Look at this excellent starter repository for an example of getting Laravel + React working by Spharian.
To render your React code inside your Laravel, set your React files’ source inside the index.blade.php body tag, by adding the following for example:
<script src=""></script>
.NET
Using the ReactJS.NET framework, you can easily introduce React into your .NET application.
Install the ReactJS.NET package to your Visual Studio IDE via the NuGET package manager for .NET.
Search the available packages for ‘ReactJS.NET (MVC 4 and 5)’ and install. You will now be able to use any .jsx extension code in your asp.net app.
Add a new controller to your project to get started with React + .NET, and select “Empty MVC Controller” for your template. Once it is created, right click on return View() and add a new view with the following details:
- View name: Index
- View Engine: Razor (CSHTML)
- Create a strongly-typed view: Unticked
- Create as a partial view: Unticked
- Use a layout or master page: Unticked
Now you can replace the default code with the following:
@{
Layout = null;
}
<html>
<head>
<title>Hello React</title>
</head>
<body>
<div id="content"></div>
<script src="http://ift.tt/1WikzFG"></script>
<script src="http://ift.tt/1Wikxxp"></script>
<script src="@Url.Content("~/Scripts/Example.jsx")"></script>
</body>
</html>
Now we need to create the Example.jsx referenced above, so create the file in your project and add your JSX as follows:
var CommentBox = React.createClass({
render: function() {
return (
<div className="commentBox">
Hello, world! I am a CommentBox.
</div>
);
}
});
ReactDOM.render(
<CommentBox />,
document.getElementById('content')
);
Now if you click Play in your Visual Studio IDE, you should see the Hello World comment box example.
Here’s a detailed tutorial on writing a component for asp.net.
Rails
By using react-rails, you can easily add React to any Rails (3.2+) application. To get started, just add the gem:
gem 'react-rails', '~> 1.7.0'
and install:
bundle install
Now you can run the installation script:
rails g react:install
This will result in two things:
- A
components.jsmanifest file inapp/assets/javascripts/components/; this is where you will put all your components code. - Adding the following to your
application.js:
//= require react //= require react_ujs //= require components
Now .jsx code will be rendering, and you can add a block of React to your template, for example:
<%= react_component('HelloMessage', name: 'John') %>
<!-- becomes: -->
<div data-react-class="HelloMessage" data-react-props="{"name":"John"}"></div>
Ruby JSX
Babel is at the heart of the Ruby implementation of the react-rails gem, and can be configured as so:
config.react.jsx_transform_options = {
blacklist: ['spec.functionName', 'validation.react', 'strict'], # default options
optional: ["transformerName"], # pass extra babel options
whitelist: ["useStrict"] # even more options
}
Once react-rails is installed into your project, restart your server and any .js.jsx files will be transformed in your asset pipeline.
For more information on react-rails, go to the official documentation.
Python
To install python-react, use pip like so:
pip install react
You can now render React code with a Python app by providing the path to your .jsx components and serving the app with a render server. Usually this is a separate Node.js process.
To run a render server, follow this easy short guide.
Now you can start your server as so:
node render_server.js
Start your python application:
python app.py
And load up http://127.0.0.1:5000 in a browser to see your React code rendering.
Django
Add react to your INSTALLED_APPS and provide some configuration as so:
INSTALLED_APPS = (
# ...
'react',
)
REACT = {
'RENDER': not DEBUG,
'RENDER_URL': 'http://ift.tt/2buaiTA',
}
Meteor
To add React to your meteor project, do so via:
meteor npm install --save react react-dom
Then in client/main.jsx add the following for example:
import React from 'react';
import { Meteor } from 'meteor/meteor';
import { render } from 'react-dom';
import App from '../imports/ui/App.jsx';
Meteor.startup(() => {
render(<App />, document.getElementById('render-target'));
});
This is instantiating an App React component, which you will define in imports/ui/App.jsx, for example:
import React, { Component } from 'react';
import Headline from './Headline.jsx';
// The App component - represents the whole app
export default class App extends Component {
getHeadlines() {
return [
{ _id: 1, text: 'Legalisation of medical marijuana goes worldwide!' },
{ _id: 2, text: 'Matt Brown goes inside the cult of scientology' },
{ _id: 3, text: 'The deep web: A criminals dream or fascinating freedom?' },
];
}
renderHeadlines() {
return this.getHeadlines().map((headline) => (
<Headline key={headline._id} headline={headline} />
));
}
render() {
return (
<div className="container">
<header>
<h1>The latest headlines</h1>
</header>
<ul>
{this.renderHeadlines()}
</ul>
</div>
);
}
}
Inside the Headline.jsx, you use the following code:
import React, { Component, PropTypes } from 'react';
// Headline component - this will display a single news headline item from a iterated array
export default class Headline extends Component {
render() {
return (
<li>{this.props.headline.text}</li>
);
}
}
Headline.propTypes = {
// This component gets the headline to display through a React prop.
// We can use propTypes to indicate it is required
headline: PropTypes.object.isRequired,
};
Meteor is ready for React and has official documentation.
No More
An important point to note: When using Meteor with React, the default templating system is no longer used as it is defunct due to React being in the project.
So instead of using or Template.templateName for helpers and events in your JS, you will define everything in your View components, which are all subclasses of React.component.
Conclusion
React can be used in practically any language which utilises an HTML presentation layer. The benefits of React can be fully exploited by a plethora of potential software products.
React makes the UI View layer become component-based. Working logically with any stack means that we have a universal language for interface that designers across all facets of web development can utilise.
React unifies our projects’ interfaces, branding and general contingency across all deployments, no matter the device or platform restraints. Also in terms of freelance, client-based work or internally inside large organisations, React ensures reusable code for your projects.
You can create your own bespoke libraries of components and get working immediately inside new projects or renovate old ones, creating fully reactive isometric application interfaces quickly and easily.
React is a significant milestone in web development, and it has the potential to become an essential tool in any developer’s collection. Don’t get left behind.
Building a WordPress-Powered Front End With the WP REST API and AngularJS: Building a Custom Directive for Post Listing
Building a WordPress-Powered Front End With the WP REST API and AngularJS: Building a Custom Directive for Post Listing
Introducing AngularJS Directives
Planning the Custom AngularJS Directive for Post Listing
Building a Custom Directive for Post Listing
Directive Declaration
Creating a Custom Template for the Directive
What’s Up Next?
In the previous part of the series, we bootstrapped our AngularJS application, configured routing for different views, and built services around routes for posts, users, and categories. Using these services, we are now finally able to fetch data from the server to power the front end.
In this part of the series, we will be working towards building a custom AngularJS directive for the post listing feature. In the current part of the series, we will:
- introduce ourselves to AngularJS directives and why we should create one
- plan the directive for the post listing feature and the arguments it will take
- create a custom AngularJS directive for post listing along with its template
So let’s start by introducing ourselves to AngularJS directives and why we need them.
Introducing AngularJS Directives
Directives in AngularJS are a way to modify the behavior of HTML elements and to reuse a repeatable chunk of code. They can be used to modify the structure of an HTML element and its children, and thus they are a perfect way to introduce custom UI widgets.
While analyzing wireframes in the first part of the series, we noted that the post listing feature is being used in three views, namely:
- Post listing
- Author profile
- Category posts listing
So instead of writing separate functionality to list posts on all of these three pages, we can create a custom AngularJS directive that contains business logic to retrieve posts using the services we created in the earlier part of this series. Apart from business logic, this directive will also contain the rendering logic to list posts on certain views. It’s also in this directive that the functionality for post pagination and retrieving posts on certain criteria will be defined.
Hence, creating a custom AngularJS directive for the post listing feature allows us to define the functionality only in one place, and this will make it easier for us in the future to extend or modify this functionality without having to change the code in all three instances where it’s being used.
Having said that, let’s begin coding our custom directive for the post listing feature.
Planning the Custom AngularJS Directive for Post Listing
Before we begin writing any code for building the directive for the post listing feature, let’s analyze the functionality that’s needed in the directive.
At the very basic level, we need a directive that we could use on our views for post listing, author profile, and the category page. This means that we will be creating a custom UI widget (or a DOM marker) that we place in our HTML, and AngularJS will take care of the rest depending upon what options we provide for that particular instance of the directive.
Hence, we will be creating a custom UI widget identified by the following tag:
<post-listing></post-listing>
But we also need this directive to be flexible, i.e. to take arguments as input and act accordingly. Consider the user profile page where we only want posts belonging to that specific user to show up or the category page where posts belonging to that category will be listed. These arguments can be provided in the following two ways:
- In the URL as parameters
- Directly to the directive as an attribute value
Providing arguments in the URL seems native to the API as we are already familiar with doing so. Hence a user could retrieve a set of posts belonging to a specific user in the following way:
http://ift.tt/2bGquS3
The above functionality can be achieved by using the $routeParams service provided by AngularJS. This is where we could access parameters provided by the user in the URL. We have already looked into it while registering routes in the previous part of the series.
As for providing arguments directly to the directive as an attribute value, we could use something like the following:
<post-listing post-args="{author=1}"></post-listing>
The post-args attribute in the above snippet takes arguments for retrieving a specific set of posts, and currently it’s taking the author ID. This attribute can take any number of arguments for retrieving posts as supported by the /wp/v2/posts route. So if we were to retrieve a set of posts authored by a user having an ID of 1 and belonging to a category of ID 10, we could do something like the following:
<post-listing post-args="{author=1, filter[cat]=10}"></post-listing>
The filter[cat] parameter in the above code is used to retrieve a set of posts belonging to a certain category.
Pagination is also an essential feature when working with post listing pages. The directive will handle post pagination, and this feature will be driven by the values of the X-WP-Total and X-WP-TotalPages headers as returned by the server along with the response body. Hence, the user will be able to navigate back and forth between the previous and next sets of posts.
Having decided the nitty gritty of the custom directive for post listing, we now have a fairly solid foundation to begin writing the code.
Building a Custom Directive for Post Listing
Building a directive for the post listing feature includes two steps:
- Create the business logic for retrieving posts and handling other stuff.
- Create a rendering view for these posts to show up on the page.
The business logic for our custom directive will be handled in the directive declaration. And for rendering data on the DOM, we will create a custom template for listing posts. Let’s start with the directive declaration.
Directive Declaration
Directives in AngularJS can be declared for a module with the following syntax:
/**
* Creating a custom directive for posts listing
*/
quiescentApp.directive( 'postListing', [function() {
return {
};
}] );
Here we are declaring a directive on our module using the .directive() method that’s available in the module. The method takes the name of the directive as the first argument, and this name is closely linked with the name of the element’s tag. Since we want our HTML element to be <post-listing></post-listing>, we provide a camel-case representation of the tag name. You can learn more about this normalization process performed by AngularJS to match directive names in the official documentation.
The notation we are using in the above code for declaring our directive is called safe-style of dependency injection. And in this notation, we provide an array of dependencies as the second argument that will be needed by the directive. Currently, we haven’t defined any dependencies for our custom directive. But since we need the Posts service for retrieving posts (that we created in the previous part of the series) and the native AngularJS’s $routeParams and $location services for accessing URL parameters and the current path, we define them as follows:
/**
* Creating a custom directive for posts listing
*/
quiescentApp.directive( 'postListing', ['$routeParams', '$location', 'Posts', function( $routeParams, $location, Posts ) {
return {
restrict: 'E',
scope: {
postArgs: '='
},
link: function( $scope, $elem, $attr ) {
}
};
}] );
These dependencies are then made available to the function which is defined as the last element of the array. This function returns an object containing directive definition. Currently, we have two properties in the directive definition object, i.e. restrict and link.
The restrict option defines the way we use directive in our code, and there can be four possible values to this option:
A: For using the directive as an attribute on an existing HTML element.E: For using the directive as an element name.C: For using the directive as a class name.M: For using the directive as an HTML comment.
The restrict option can also accept any combination of the above four values.
Since we want our directive to be a new element <post-listing></post-listing>, we set the restrict option to E. If we were to define the directive using the attributes on a pre-existing HTML element, then we could have set this option to A. In that case, we could use <div post-listing></div> to define the directive in our HTML code.
The second scope property is used to modify the scope of the directive. By default, the value of the scope property is false, meaning that the scope of the directive is the same as its parent’s. When we pass it an object, an isolated scope is created for the directive and any data that needs to be passed to the directive by its parent is passed through HTML attributes. This is what we are doing in our code, and the attribute we are using is post-args, which gets normalized into postArgs.
The postArgs property in the scope object can accept any of the following three values:
=: Meaning that the value passed into the attribute would be treated as an object.@: Meaning that the value passed into the attribute would be treated as a plain string.&: Meaning that the value passed into the attribute would be treated as a function.
Since we have chosen to use the = value, any value that gets passed into the post-args attribute would be treated as a JSON object, and we could use that object as an argument for retrieving posts.
The third property, link, is used to define a function that is used to manipulate the DOM and define APIs and functions that are necessary for the directive. This function is where all the logic of the directive is handled.
The link function accepts arguments for the scope object, the directive’s HTML element, and an object for attributes defined on the directive’s HTML element. Currently, we are passing two arguments $scope and $elem for the scope object and the HTML element respectively.
Let’s define some variable on the $scope property that we will be using to render the post listing feature on the DOM.
/**
* Creating a custom directive for posts listing
*/
quiescentApp.directive( 'postListing', ['$routeParams', '$location', 'Posts', function( $routeParams, $location, Posts ) {
return {
restrict: 'E',
scope: {
postArgs: '='
},
link: function( $scope, $elem, $attr ) {
// defining variables on the $scope object
$scope.posts = [];
$scope.postHeaders = {};
$scope.currentPage = $routeParams.page ? Math.abs( $routeParams.page ) : 1;
$scope.nextPage = null;
$scope.previousPage = null;
$scope.routeContext = $location.path();
}
};
}] );
Hence we have defined six properties on the $scope object that we could access in the DOM. These properties are:
$posts: An array for holding post objects that will be returned by the server.$postHeaders: An object for holding the headers that will be returned by the server along with the response body. We will use these for handling navigation.$currentPage: An integer variable holding the current page number.$previousPage: A variable holding the previous page number.$nextPage: A variable holding the next page number.$routeContext: For accessing the current path using the$locationservice.
The postArgs property that we defined earlier for HTML attributes will already be available on the $scope object inside the directive.
Now we are ready to make a request to the server using the Posts service for retrieving posts. But before that, we must take into account the arguments provided by the user as URL parameters as well as the parameters provided in the post-args attribute. And for that purpose, we will create a function that uses the $routeParams service to extract URL parameters and merge them with the arguments provided through the post-args attribute:
/**
* Creating a custom directive for posts listing
*/
quiescentApp.directive( 'postListing', ['$routeParams', '$location', 'Posts', function( $routeParams, $location, Posts ) {
return {
restrict: 'E',
scope: {
postArgs: '='
},
link: function( $scope, $elem, $attr ) {
// defining variables on the $scope object
$scope.posts = [];
$scope.postHeaders = {};
$scope.currentPage = $routeParams.page ? Math.abs( $routeParams.page ) : 1;
$scope.nextPage = null;
$scope.previousPage = null;
$scope.routeContext = $location.path();
// preparing query arguments
var prepareQueryArgs = function() {
var tempParams = $routeParams;
delete tempParams.id;
return angular.merge( {}, $scope.postArgs, tempParams );
};
}
};
}] );
The prepareQueryArgs() method in the above code uses the angular.merge() method, which extends the $scope.postArgs object with the $routeParams object. But before merging these two objects, it first deletes the id property from the $routeParams object using the delete operator. This is necessary since we will be using this directive on category and user views, and we don’t want the category and user IDs to get falsely interpreted as the post ID.
Having prepared query arguments, we are finally ready to make a call to the server and retrieve posts, and we do so with the Posts.query() method, which takes two arguments:
- An object containing arguments for making the query.
- A callback function that executes after the query has been completed.
So we will use the prepareQueryArgs() function for preparing an object for query arguments, and in the callback function, we set the values of certain variables on the $scope property:
// make the request and query posts
Posts.query( prepareQueryArgs(), function( data, headers ) {
$scope.posts = data;
$scope.postHeaders = headers();
$scope.previousPage = ( ( $scope.currentPage + 1 ) > $scope.postHeaders['x-wp-totalpages'] ) ? null : ( $scope.currentPage + 1 );
$scope.nextPage = ( ( $scope.currentPage - 1 ) > 0 ) ? ( $scope.currentPage - 1 ) : null;
});
The callback function gets passed two arguments for the response body and the response headers. These are represented by the data and headers arguments respectively.
The headers argument is a function that returns an object containing response headers by the server.
The remaining code is pretty self-explanatory as we are setting the value of the $scope.posts array. For setting the values of the $scope.previousPage and $scope.nextPage variables, we are using the x-wp-totalpages property in the postHeaders object.
And now we are ready to render this data on the front end using a custom template for our directive.
Creating a Custom Template for the Directive
The last thing we need to do in order to make our directive work is to make a separate template for post listing and link it to the directive. For that purpose, we need to modify the directive declaration and include a templateUrl property like the following:
/**
* Creating a custom directive for posts listing
*/
quiescentApp.directive( 'postListing', ['$routeParams', '$location', 'Posts', function( $routeParams, $location, Posts ) {
return {
restrict: 'E',
scope: {
postArgs: '='
},
templateUrl: 'views/directive-post-listing.html',
link: function( $scope, $elem, $attr ) {
}
};
}] );
This templateUrl property in the above code refers to a file named directive-post-listing.html in the views directory. So create this file in the views folder and paste in the following HTML code:
<!-- post listing starts -->
<article class="post-entry">
<h2 class="post-title"><a href="post-single.html">Good design is a lot like clear thinking made visual.</a></h2>
<figure class="post-thumbnail">
<img src="img/img-712-348.jpg" alt="Featured Image">
</figure>
<p class="post-meta">
By <a href="author.html">Bilal Shahid</a> in <a href="category.html">Quotes</a>
</p>
<div class="post-content">
<p>Created days forth. Dominion. Subdue very hath spirit us sixth fish creepeth also. First meat one forth above. You'll Fill for. Can't evening one lights won't. Great of make firmament image. Life his beginning blessed lesser meat spirit blessed seas created green great beginning can't doesn't void moving. Subdue evening make spirit lesser greater all living green firmament winged saw tree one divide wherein divided shall dry very lesser saw, earth the. Light their the.</p>
</div>
</article>
<!-- post listing ends -->
<!-- pagination links start -->
<div class="post-pagination">
<a href="#" class="button">Older Posts</a>
<a href="#" class="button">Newer Posts</a>
</div>
<!-- pagination links end -->
This is very basic HTML code representing a single post entry and post pagination. I’ve copied it from the views/listing.html file. We will use some AngularJS directives, including ng-repeat, ng-href, ng-src, and ng-bind-html, to display the data that currently resides in the $scope property of the directive.
Modify the HTML code to the following:
<!-- post listing starts -->
<article class="post-entry" ng-repeat="post in posts">
<h2 class="post-title"><a ng-href="#/posts/"></a></h2>
<figure class="post-thumbnail" ng-show="post.quiescent_featured_image">
<img ng-src="" alt="Featured Image">
</figure>
<p class="post-meta">
By <a ng-href="#/users/"></a>
in <a ng-href="#/categories/" ng-repeat="category in post.quiescent_categories"></a>
</p>
<div class="post-content" ng-bind-html="post.excerpt.rendered"></div>
</article>
<!-- post listing ends -->
The above code uses the ng-repeat directive to iterate through the $scope.posts array. Any property that is defined on the $scope object in the directive declaration is available directly in the template. Hence, we refer to the $scope.posts array directly as posts in the template.
By using the ng-repeat directive, we ensure that the article.post-entry container will be repeated for each post in the posts array and each post is referred to as post in the inner loop. This post object contains data in the JSON format as returned by the server, containing properties like the post title, post ID, post content, and the featured image link, which is an additional field added by the companion plugin.
In the next step, we replace values like the post title, the post link, and the featured image link with properties in the post object.
For the pagination, replace the previous code with the following:
<!-- pagination links start -->
<div class="post-pagination">
<a ng-href="#?page=" class="button" ng-class="{'disabled': !nextPage}">Newer Posts</a>
<a ng-href="#?page=" class="button" ng-class="{'disabled': !previousPage}">Older Posts</a>
</div>
<!-- pagination links end -->
We first access the routeContext property, which we defined in our directive declaration, and suffix it with the ?page= parameter and use the values of the nextPage and previousPage variables to navigate back and forth between posts. We also check to see if the next page or the previous page link is not null, else we add a .disabled class to the button that is provided by Zurb Foundation.
Now that we've finished the directive, it's time to test it. And we do it by placing a <post-listing></post-listing> tag in our HTML, ideally right above the <footer></footer> tag. Doing so means that a post listing will appear above the page footer. Don’t worry about the formatting and styles as we will deal with them in the next part of the series.
So that’s pretty much it for creating a custom AngularJS directive for the post listing feature.
What’s Up Next?
In the current part of the series about creating a front end with the WP REST API and AngularJS, we built a custom AngularJS directive for the post listing feature. This directive uses the Posts service that we created in the earlier part of the series. The directive also takes user input in the form of an HTML attribute and through URL parameters.
In the concluding part of the series, we will begin working on the final piece of our project, i.e. controllers for posts, users, and categories, and their respective templates.
Friday, August 19, 2016
Let's Go: Object-Oriented Programming in Golang
Let's Go: Object-Oriented Programming in Golang
Go is a strange mix of old and new ideas. It has a very refreshing approach where it isn't afraid to throw away established notions of "how to do things". Many people are not even sure if Go is an object-oriented language. Let me put that to rest right now. It is!
In this tutorial you'll learn about all the intricacies of object-oriented design in Go, how the pillars of object-oriented programming like encapsulation, inheritance, and polymorphism are expressed in Go, and how Go compares to other languages.
The Go Design Philosophy
Go's roots are based on C and more broadly on the Algol family. Ken Thompson half-jokingly said that Rob Pike, Robert Granger and himself got together and decided they hate C++. Whether it's a joke or not, Go is very different from C++. More on that later. Go is about ultimate simplicity. This is explained in detail by Rob Pike in Less is exponentially more.
Go vs. Other Languages
Go has no classes, no objects, no exceptions, and no templates. It has garbage collection and built-in concurrency. The most striking omission as far as object-oriented is concerned is that there is no type hierarchy in Go. This is in contrast to most object-oriented languages like C++, Java, C#, Scala, and even dynamic languages like Python and Ruby.
Go Object-Oriented Language Features
Go has no classes, but it has types. In particular, it has structs. Structs are user-defined types. Struct types (with methods) serve similar purposes to classes in other languages.
Structs
A struct defines state. Here is a Creature struct. It has a Name field and a boolean flag called Real, which tells us if it's a real creature or an imaginary creature. Structs hold only state and no behavior.
type Creature struct {
Name string
Real bool
}
Methods
Methods are functions that operate on particular types. They have a receiver clause that mandates what type they operate on. Here is a Dump() method that operates on Creature structs and prints their state:
func (c Creature) Dump() {
fmt.Printf("Name: '%s', Real: %t\n", c.Name, c.Real)
}
This is an unusual syntax, but it is very explicit and clear (unlike the implicit "this" or Python's confusing "self").
Embedding
You can embed anonymous types inside each other. If you embed a nameless struct then the embedded struct provides its state (and methods) to the embedding struct directly. For example, the FlyingCreature has a nameless Creature struct embedded in it, which means a FlyingCreature is a Creature.
type FlyingCreature struct {
Creature
WingSpan int
}
Now, if you have an instance of a FlyingCreature, you can access its Name and Real attributes directly.
dragon := &FlyingCreature{
Creature{"Dragon", false, },
15,
}
fmt.Println(dragon.Name)
fmt.Println(dragon.Real)
fmt.Println(dragon.WingSpan)
Interfaces
Interfaces are the hallmark of Go's object-oriented support. Interfaces are types that declare sets of methods. Similarly to interfaces in other languages, they have no implementation.
Objects that implement all the interface methods automatically implement the interface. There is no inheritance or subclassing or "implements" keyword. In the following code snippet, type Foo implements the Fooer interface (by convention, Go interface names end with "er").
type Fooer interface {
Foo1()
Foo2()
Foo3()
}
type Foo struct {
}
func (f Foo) Foo1() {
fmt.Println("Foo1() here")
}
func (f Foo) Foo2() {
fmt.Println("Foo2() here")
}
func (f Foo) Foo3() {
fmt.Println("Foo3() here")
}
Object-Oriented Design: The Go Way
Let's see how Go measures up against the pillars of object-oriented programming: encapsulation, inheritance, and polymorphism. Those are features of class-based programming languages, which are the most popular object-oriented programming languages.
At the core, objects are language constructs that have state and behavior that operates on the state and selectively exposes it to other parts of the program.
Encapsulation
Go encapsulates things at the package level. Names that start with a lowercase letter are only visible within that package. You can hide anything in a private package and just expose specific types, interfaces, and factory functions.
For example, here to hide the Foo type above and expose just the interface you could rename it to lower case foo and provide a NewFoo() function that returns the public Fooer interface:
type foo struct {
}
func (f foo) Foo1() {
fmt.Println("Foo1() here")
}
func (f foo) Foo2() {
fmt.Println("Foo2() here")
}
func (f foo) Foo3() {
fmt.Println("Foo3() here")
}
func NewFoo() Fooer {
return &Foo{}
}
Then code from another package can use NewFoo() and get access to a Fooer interface implemented by the internal foo type:
f := NewFoo() f.Foo1() f.Foo2() f.Foo3()
Inheritance
Inheritance or subclassing was always a controversial issue. There are many problems with implementation inheritance (as opposed to interface inheritance). Multiple inheritance as implemented by C++ and Python and other languages suffers from the deadly diamond of death problem, but even single inheritance is no picnic with the fragile base-class problem.
Modern languages and object-oriented thinking now favor composition over inheritance. Go takes it to heart and doesn't have any type hierarchy whatsoever. It allows you to share implementation details via composition. But Go, in a very strange twist (that probably originated from pragmatic concerns), allows anonymous composition via embedding.
For all intents and purposes, composition by embedding an anonymous type is equivalent to implementation inheritance. An embedded struct is just as fragile as a base class. You can also embed an interface, which is equivalent to inheriting from an interface in languages like Java or C++. It can even lead to a runtime error that is not discovered at compile time if the embedding type doesn't implement all the interface methods.
Here SuperFoo embeds the Fooer interface, but doesn't implement its methods. The Go compiler will happily let you create a new SuperFoo and call the Fooer methods, but will obviously fail at runtime. This compiles:
type SuperFooer struct {
Fooer
}
func main() {
s := SuperFooer{}
s.Foo2()
Running this program results in a panic:
panic: runtime error: invalid memory address or nil pointer dereference [signal 0xb code=0x1 addr=0x28 pc=0x2a78] goroutine 1 [running]: panic(0xde180, 0xc82000a0d0) /usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6 main.main() /Users/gigi/Documents/dev/go/src/http://ift.tt/2bCxfoo +0x48 exit status 2 Process finished with exit code 1
Polymorphism
Polymorphism is the essence of object-oriented programming: the ability to treat objects of different types uniformly as long as they adhere to the same interface. Go interfaces provide this capability in a very direct and intuitive way.
Here is an elaborate example where multiple creatures (and a door!) that implement the Dumper interface are created and stored in a slice and then the Dump() method is called for each one. You'll notice different styles of instantiating the objects too.
package main
import "fmt"
type Creature struct {
Name string
Real bool
}
func Dump(c*Creature) {
fmt.Printf("Name: '%s', Real: %t\n", c.Name, c.Real)
}
func (c Creature) Dump() {
fmt.Printf("Name: '%s', Real: %t\n", c.Name, c.Real)
}
type FlyingCreature struct {
Creature
WingSpan int
}
func (fc FlyingCreature) Dump() {
fmt.Printf("Name: '%s', Real: %t, WingSpan: %d\n",
fc.Name,
fc.Real,
fc.WingSpan)
}
type Unicorn struct {
Creature
}
type Dragon struct {
FlyingCreature
}
type Pterodactyl struct {
FlyingCreature
}
func NewPterodactyl(wingSpan int) *Pterodactyl {
pet := &Pterodactyl{
FlyingCreature{
Creature{
"Pterodactyl",
true,
},
wingSpan,
},
}
return pet
}
type Dumper interface {
Dump()
}
type Door struct {
Thickness int
Color string
}
func (d Door) Dump() {
fmt.Printf("Door => Thickness: %d, Color: %s", d.Thickness, d.Color)
}
func main() {
creature := &Creature{
"some creature",
false,
}
uni := Unicorn{
Creature{
"Unicorn",
false,
},
}
pet1 := &Pterodactyl{
FlyingCreature{
Creature{
"Pterodactyl",
true,
},
5,
},
}
pet2 := NewPterodactyl(8)
door := &Door{3, "red"}
Dump(creature)
creature.Dump()
uni.Dump()
pet1.Dump()
pet2.Dump()
creatures := []Creature{
*creature,
uni.Creature,
pet1.Creature,
pet2.Creature}
fmt.Println("Dump() through Creature embedded type")
for _, creature := range creatures {
creature.Dump()
}
dumpers := []Dumper{creature, uni, pet1, pet2, door}
fmt.Println("Dump() through Dumper interface")
for _, dumper := range dumpers {
dumper.Dump()
}
}
Conclusion
Go is a bona fide object-oriented programming language. It enables object-based modeling and promotes the best practice of using interfaces instead of concrete type hierarchies. Go made some unusual syntactic choices, but overall working with types, methods, and interfaces feels simple, lightweight, and natural.
Embedding is not very pure, but apparently pragmatism was at work, and embedding was provided instead of only composition by name.
Subscribe to:
Comments (Atom)