This is the eleventh post in my SCons series. This post starts exploring ways to work with protocol buffer files in a SCons project.
Protocol buffers are a structured-data-serialization mechanism from Google. This is not a tutorial on protocol buffers. I will use the address book project example that appears in the official protocol buffers tutorial.
When using protocol buffers, you write .proto files to describe your structured data. You use the protoc compiler to generate C++ and Python code that allows you to serialize, load, and manipulate your protocol buffers data.
Out of the box, SCons does not know how to compile .proto files into C++ and Python code. The purpose of this post is to start exploring ways to integrate protocol buffers in the build process. The first iteration is based on manual “integration”.
The final result is available on my GitHub scons-series repository.
Converting the AddressBook project to Protocol Buffers
As luck (or early planning) would have it, the silly C++ address book program I’m using throughout the series is surprisingly similar to the C++ address book program in Google’s Protocol Buffers tutorial!
As a first step, I replace my address book and writer code from the previous episodes with the code from Google’s tutorial:
- My addressbook.h and addressbook.cc are no longer relevant. They are replaced with addressbook.proto that represents pretty much the same data structure I had before.
- The writer.cc which did very little is also replaced. The new writer.cpp is a bit less silly. It supports reading multiple address book entries from the user, and writing the entire thing to a address book proto file passed as an argument to the program.
- A new reader.cc is added. It’s a program that simply lists the contents of the address book in the proto file received as an argument.
For convenience, here’s the addressbook.proto file that describes the Person and AddressBook data structures:
// Source: https://developers.google.com/protocol-buffers/docs/cpptutorial
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
message AddressBook {
repeated Person person = 1;
}
Now I use the protoc compiler to manually generate the C++ address book code:
itamar@legolas sconseries (episodes/10-protobuf) $ rm -r build/ itamar@legolas sconseries (episodes/10-protobuf) $ ls AddressBook/ SConscript addressbook.proto itamar@legolas sconseries (episodes/10-protobuf) $ protoc -I. --cpp_out . AddressBook/addressbook.proto itamar@legolas sconseries (episodes/10-protobuf) $ ls AddressBook/ SConscript addressbook.pb.cc addressbook.pb.h addressbook.proto
The addressbook.pb.cc and addressbook.pb.h files are the generated code that replaces the previous AddressBook and Person classes.
To build the project with SCons, I just update the AddressBook/SConscript file to refer to the .pb.cc file:
"""AddressBook library SConscript script"""
Import('*')
Lib('addressbook', 'addressbook.pb.cc')
Additionally, I need to link the protobuf library with the Writer & Reader program targets:
"""AddressBook Writer SConscript script"""
Import('*')
Prog('writer', 'writer.cc', with_libs='AddressBook::addressbook', LIBS=['protobuf'])
"""AddressBook Writer SConscript script"""
Import('*')
Prog('reader', 'reader.cc', with_libs='AddressBook::addressbook', LIBS=['protobuf'])
And now I can build the project by running scons, just as in previous episodes:
itamar@legolas sconseries (episodes/10-protobuf) $ source mode debug (debug) itamar@legolas gh_sconseries (episodes/10-protobuf) $ scons scons: Reading SConscript files ... scons: Using active flavor "debug" from your environment scons: + Processing flavor debug ... scons: |- First pass: Reading module AddressBook ... scons: |- First pass: Reading module Reader ... scons: |- First pass: Reading module Writer ... scons: |- Second pass: Reading module AddressBook ... scons: |- Second pass: Reading module Reader ... scons: |- Second pass: Reading module Writer ... scons: done reading SConscript files. scons: Building targets ... clang++ -o build/debug/Reader/reader.o -c -std=c++11 -Wall -fvectorize -fslp-vectorize -g -DDEBUG -Ibuild/debug build/debug/Reader/reader.cc clang++ -o build/debug/Reader/reader build/debug/Reader/reader.o build/debug/AddressBook/libaddressbook.a -lprotobuf clang++ -o build/debug/Writer/writer.o -c -std=c++11 -Wall -fvectorize -fslp-vectorize -g -DDEBUG -Ibuild/debug build/debug/Writer/writer.cc clang++ -o build/debug/Writer/writer build/debug/Writer/writer.o build/debug/AddressBook/libaddressbook.a -lprotobuf Install file: "build/debug/Reader/reader" as "build/debug/bin/Reader.reader" Install file: "build/debug/Writer/writer" as "build/debug/bin/Writer.writer" scons: done building targets. (debug) itamar@legolas gh_sconseries (episodes/10-protobuf) $ Writer.writer foo.adbook foo.adbook: File not found. Creating a new file. Enter person ID number: 123 Enter name: Oogi Enter email address (blank for none): oogi@oogi.oogi Enter a phone number (or leave blank to finish): 12321 Is this a mobile, home, or work phone? work Enter a phone number (or leave blank to finish): (debug) itamar@legolas gh_sconseries (episodes/10-protobuf) $ Reader.reader foo.adbook Person ID: 123 Name: Oogi E-mail address: oogi@oogi.oogi Work phone #: 12321
In case you’re curious about how the serialized address book protobuf looks like, here it is:
(debug) itamar@legolas sconseries (episodes/10-protobuf) $ cat -vent foo.adbook
1 $
2 #$
3 ^DOogi^P{^Z^Noogi@oogi.oogi"^I$
4 ^E12321^P^B
Summary
That’s it for the “manual approach” to using protocol buffers in a SCons project.
It’s not much, actually. I didn’t really integrate protobuf compilation into SCons yet. I compiled my .proto file into C++ code, and built that code with SCons just like before.
Consider this a preview for the future episodes:
- Actually using SCons to compile protobuf files (probably in two parts).
- Integrating the SCons proto builder into my shortcuts system.
- Expand the integration to handle generated Python code.
This episode’s code is available on my GitHub scons-series repository. Feel free to use / fork / modify. If you do, I’d appreciate it if you share back improvements.
Leave a Reply