Supported protocol buffer and Arrow data types

This document describes the supported protocol buffer and Arrow data types for each respective BigQuery data type. Before reading this document, read Overview of the BigQuery Storage Write API .

Supported protocol buffer data types

The following table shows the supported data types in protocol buffers and the corresponding input format in BigQuery:

BigQuery data type

Supported protocol buffer types

BOOL

bool , int32 , int64 , uint32 , uint64 , google.protobuf.BoolValue

BYTES

bytes , string , google.protobuf.BytesValue

DATE

int32 (preferred), int64 , string

The value is the number of days since the Unix epoch (1970-01-01). The valid range is -719162 (0001-01-01) to 2932896 (9999-12-31).

DATETIME , TIME

string

The value must be a DATETIME or TIME literal.

int64

Use the CivilTimeEncoder class to perform the conversion.

FLOAT

double , float , google.protobuf.DoubleValue , google.protobuf.FloatValue

GEOGRAPHY

string

The value is a geometry in either WKT or GeoJson format.

INTEGER

int32 , int64 , uint32 , enum , google.protobuf.Int32Value , google.protobuf.Int64Value , google.protobuf.UInt32Value

JSON

string

NUMERIC , BIGNUMERIC

int32 , int64 , uint32 , uint64 , double , float , string

bytes , google.protobuf.BytesValue

Use the BigDecimalByteStringEncoder class to perform the conversion.

STRING

string , enum , google.protobuf.StringValue

TIME

string

The value must be a TIME literal .

TIMESTAMP

int64 (preferred), int32 , uint32 , google.protobuf.Timestamp

The value is given in microseconds since the Unix epoch (1970-01-01).

INTERVAL

string , google.protobuf.Duration

The string value must be an INTERVAL literal .

RANGE<T>

message

A nested message type in the proto with two fields, start and end , where both fields must be of the same supported protocol buffer type that corresponds to a BigQuery data type T . T must be one of DATE , DATETIME , or TIMESTAMP . If a field ( start or end ) is not set in the proto message, it represents an unbounded boundary. In the following example, f_range_date represents a RANGE column in a table. Since the end field is not set in the proto message, the end boundary of this range is unbounded.

 {
  f_range_date: {
    start: 1
  }
}

REPEATED FIELD

array

An array type in the proto corresponds to a repeated field in BigQuery.

RECORD

message

A nested message type in the proto corresponds to a record field in BigQuery.

Supported Apache Arrow data types

The following table shows the supported data types in Apache Arrow and the corresponding input format in BigQuery.

BigQuery data type

Supported Apache Arrow types

Supported type parameters

BOOL

Boolean

BYTES

Binary

DATE

Date

unit = Day

String , int32

DATETIME

Timestamp

unit = MICROSECONDS

timezone is empty

FLOAT

FloatingPoint

Precision in {SINGLE, DOUBLE}

GEOGRAPHY

Utf8

The value is a geometry in either WKT or GeoJson format.

INTEGER

int

bitWidth in {8, 16, 32, 64}

is_signed = false

JSON

Utf8

NUMERIC

Decimal128

You can provide a NUMERIC that has any precision or scale that's smaller than the BigQuery supported range .

BIGNUMERIC

Decimal256

You can provide a BIGNUMERIC that has any precision or scale that's smaller than the BigQuery supported range .

STRING

Utf8

TIMESTAMP

Timestamp

unit= MICROSECONDS

timezone = UTC

INTERVAL

Interval

unit in {YEAR_MONTH, DAY_TIME, MONTH_DAY_NANO}

Utf8

RANGE<T>

Struct

The Arrow Struct must have two subfields named start and end .

For the RANGE<DATE> column, the fields must be Arrow type Date with unit=Day .

For the RANGE<DATETIME> column, the fields must be the Arrow type Timestamp with unit=MICROSECONDS , without the timezone.

For the RANGE<TIMESTAMP> , the fields must be the Arrow type Timestamp with unit=MICROSECONDS , timezone=UTC .

A NULL value in any of the start and end fields will be treated as UNBOUNDED .

REPEATED FIELD

List

A NULL value must be represented by an empty list.

RECORD

Struct