博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
在Android中使用Protocol Buffers(中篇)
阅读量:6889 次
发布时间:2019-06-27

本文共 7089 字,大约阅读时间需要 23 分钟。

本文来自。

FlatBuffers 编码原理

 

FlatBuffers的Java库只提供了如下的4个类:

./com/google/flatbuffers/Constants.java./com/google/flatbuffers/FlatBufferBuilder.java./com/google/flatbuffers/Struct.java./com/google/flatbuffers/Table.java

Constants 类定义FlatBuffers中可用的基本原始数据类型的长度:

public class Constants {    // Java doesn't seem to have these.    /** The number of bytes in an `byte`. */    static final int SIZEOF_BYTE = 1;    /** The number of bytes in a `short`. */    static final int SIZEOF_SHORT = 2;    /** The number of bytes in an `int`. */    static final int SIZEOF_INT = 4;    /** The number of bytes in an `float`. */    static final int SIZEOF_FLOAT = 4;    /** The number of bytes in an `long`. */    static final int SIZEOF_LONG = 8;    /** The number of bytes in an `double`. */    static final int SIZEOF_DOUBLE = 8;    /** The number of bytes in a file identifier. */    static final int FILE_IDENTIFIER_LENGTH = 4;}

FlatBufferBuilder 用于FlatBuffers编码,它会将我们的结构化数据序列化为字节数组。我们借助于 FlatBufferBuilder 在 ByteBuffer 中放置基本数据类型的数据、数组、字符串及对象。ByteBuffer 用于处理字节序,在序列化时,它将数据按适当的字节序进行序列化,在发序列化时,它将多个字节转换为适当的数据类型。在 .fbs 文件中定义的 table 和 struct,为它们生成的Java 类会继承 TableStruct

在反序列化时,输入的ByteBuffer数据被当作字节数组,Table提供了针对字节数组的操作,生成的Java类负责对这些数据进行解释。对于FlatBuffers编码的数据,无需进行解码,只需进行解释。在编译 .fbs 文件时,每个字段在这段数据中的位置将被确定。每个字段的类型及长度将被硬编码进生成的Java类。

 

Struct 类的代码也比较简洁:

package com.google.flatbuffers;import java.nio.ByteBuffer;/// @cond FLATBUFFERS_INTERNAL/** * All structs in the generated code derive from this class, and add their own accessors. */public class Struct {  /** Used to hold the position of the `bb` buffer. */  protected int bb_pos;  /** The underlying ByteBuffer to hold the data of the Struct. */  protected ByteBuffer bb;}

整体的结构如下图:

 

在序列化结构化数据时,我们首先需要创建一个 FlatBufferBuilder ,在这个对象的创建过程中会分配或从调用者那里获取 ByteBuffer,序列化的数据将保存在这个 ByteBuffer中:

 

/**    * Start with a buffer of size `initial_size`, then grow as required.    *    * @param initial_size The initial size of the internal buffer to use.    */    public FlatBufferBuilder(int initial_size) {        if (initial_size <= 0) initial_size = 1;        space = initial_size;        bb = newByteBuffer(initial_size);    }   /**    * Start with a buffer of 1KiB, then grow as required.    */    public FlatBufferBuilder() {        this(1024);    }    /**     * Alternative constructor allowing reuse of {@link ByteBuffer}s.  The builder     * can still grow the buffer as necessary.  User classes should make sure     * to call {@link #dataBuffer()} to obtain the resulting encoded message.     *     * @param existing_bb The byte buffer to reuse.     */    public FlatBufferBuilder(ByteBuffer existing_bb) {        init(existing_bb);    }    /**     * Alternative initializer that allows reusing this object on an existing     * `ByteBuffer`. This method resets the builder's internal state, but keeps     * objects that have been allocated for temporary storage.     *     * @param existing_bb The byte buffer to reuse.     * @return Returns `this`.     */    public FlatBufferBuilder init(ByteBuffer existing_bb){        bb = existing_bb;        bb.clear();        bb.order(ByteOrder.LITTLE_ENDIAN);        minalign = 1;        space = bb.capacity();        vtable_in_use = 0;        nested = false;        finished = false;        object_start = 0;        num_vtables = 0;        vector_num_elems = 0;        return this;    }    static ByteBuffer newByteBuffer(int capacity) {        ByteBuffer newbb = ByteBuffer.allocate(capacity);        newbb.order(ByteOrder.LITTLE_ENDIAN);        return newbb;    }

下面我们更详细地分析基本数据类型数据、数组及对象的序列化过程。ByteBuffer 为小尾端的。

 

FlatBuffers编码基本数据类型

 

FlatBuffer 的基本数据类型主要包括如下这些:

 

BooleanByteShortIntLongFloatDouble

FlatBufferBuilder 提供了三组方法用于操作这些数据:

 

public void putBoolean(boolean x);    public void putByte   (byte    x);    public void putShort  (short   x);    public void putInt    (int     x);    public void putLong   (long    x);    public void putFloat  (float   x);    public void putDouble (double  x);    public void addBoolean(boolean x);    public void addByte   (byte    x);    public void addShort  (short   x);    public void addInt    (int     x);    public void addLong   (long    x);    public void addFloat  (float   x);    public void addDouble (double  x);    public void addBoolean(int o, boolean x, boolean d);    public void addByte(int o, byte x, int d);    public void addShort(int o, short x, int d);    public void addInt    (int o, int     x, int     d);    public void addLong   (int o, long    x, long    d);    public void addFloat  (int o, float   x, double  d);    public void addDouble (int o, double  x, double  d);

putXXX 那一组,直接地将一个数据放入 ByteBuffer 中,它们的实现基本如下面这样:

public void putBoolean(boolean x) {        bb.put(space -= Constants.SIZEOF_BYTE, (byte) (x ? 1 : 0));    }    public void putByte(byte x) {        bb.put(space -= Constants.SIZEOF_BYTE, x);    }    public void putShort(short x) {        bb.putShort(space -= Constants.SIZEOF_SHORT, x);    }

Boolean值会被先转为byte类型再放入 ByteBuffer。另外一点值得注意的是,数据是从 ByteBuffer 的结尾处开始放置的,space用于记录最近放入的数据的位置及剩余的空间。

addXXX(XXX x) 那一组在放入数据之前会先做对齐处理,并在需要时扩展 ByteBuffer 的容量:

 

static ByteBuffer growByteBuffer(ByteBuffer bb) {        int old_buf_size = bb.capacity();        if ((old_buf_size & 0xC0000000) != 0)  // Ensure we don't grow beyond what fits in an int.            throw new AssertionError("FlatBuffers: cannot grow buffer beyond 2 gigabytes.");        int new_buf_size = old_buf_size << 1;        bb.position(0);        ByteBuffer nbb = newByteBuffer(new_buf_size);        nbb.position(new_buf_size - old_buf_size);        nbb.put(bb);        return nbb;    }   public void pad(int byte_size) {       for (int i = 0; i < byte_size; i++) bb.put(--space, (byte) 0);   }    public void prep(int size, int additional_bytes) {        // Track the biggest thing we've ever aligned to.        if (size > minalign) minalign = size;        // Find the amount of alignment needed such that `size` is properly        // aligned after `additional_bytes`        int align_size = ((~(bb.capacity() - space + additional_bytes)) + 1) & (size - 1);        // Reallocate the buffer if needed.        while (space < align_size + size + additional_bytes) {            int old_buf_size = bb.capacity();            bb = growByteBuffer(bb);            space += bb.capacity() - old_buf_size;        }        pad(align_size);    }    public void addBoolean(boolean x) {        prep(Constants.SIZEOF_BYTE, 0);        putBoolean(x);    }    public void addInt(int x) {        prep(Constants.SIZEOF_INT, 0);        putInt(x);    }

对齐是数据存放的起始位置相对于ByteBuffer的结束位置的对齐,additional bytes被认为是不需要对齐的,且在必要的时候会在ByteBuffer可用空间的结尾处填充值为0的字节。在扩展 ByteBuffer 的空间时,老的ByteBuffer被放在新ByteBuffer的结尾处。

 

addXXX(int o, XXX x, YYY y) 这一组方法在放入数据之后,会将 vtable 中对应位置的值更新为最近放入的数据的offset。

 

public void addShort(int o, short x, int d) {        if (force_defaults || x != d) {            addShort(x);            slot(o);        }    }    public void slot(int voffset) {        vtable[voffset] = offset();    }

后面我们在分析编码对象时再来详细地了解vtable。

 

基本上,在我们的应用程序代码中不要直接调用这些方法,它们主要在构造对象时用于存储对象的基本数据类型字段。

 
 

网易云新用户大礼包:

本文来自网易云社区,经作者韩鹏飞授权发布。

转载地址:http://yrqbl.baihongyu.com/

你可能感兴趣的文章
SOAP接口
查看>>
编译安装
查看>>
IP报文头
查看>>
百度统计个人初探
查看>>
我的友情链接
查看>>
phpstorm使用
查看>>
单元测试、集成测试和系统测试的不同之处[转]
查看>>
Elasticsearch注意事项
查看>>
【数据结构】找出N个数据中最大的前k个数据(利用堆排序)
查看>>
centos7安装使用samba服务器免密码登录简单配置
查看>>
mysql中if-elesif-endif使用
查看>>
drbd状态信息详细说明
查看>>
apache详解
查看>>
闲聊 -软路由的安装
查看>>
PHP中利用COOKIE与SESSION联合实现SESSION跨域
查看>>
error:Microsoft Visual C++ 9.0 is required. Get it
查看>>
Mininal Desktop安装CentOS 6.4后编译安装Mplayer
查看>>
好马不回头策略
查看>>
函数声明后面的const用法
查看>>
CUDA中自动初始化显卡设备宏
查看>>