public class HiveInputFormatFragmenter extends HiveDataFragmenter
HiveDataFragmenter
, this class does not send the serde properties to
the accessor/resolvers. This is done to avoid memory explosion in Hawq. For
RC use together with HiveRCFileAccessor
/
HiveColumnarSerdeResolver
. For Text use together with
HiveLineBreakAccessor
/HiveStringPassResolver
. Modifier and Type | Class and Description |
---|---|
static class |
HiveInputFormatFragmenter.PXF_HIVE_INPUT_FORMATS
Defines the Hive input formats currently supported in pxf
|
static class |
HiveInputFormatFragmenter.PXF_HIVE_SERDES
Defines the Hive serializers (serde classes) currently supported in pxf
|
Modifier and Type | Field and Description |
---|---|
static int |
TOK_FILTER_DONE |
static int |
TOK_KEYS |
static int |
TOK_SERDE |
filterInFragmenter
fragments
Constructor and Description |
---|
HiveInputFormatFragmenter(InputData inputData)
Constructs a HiveInputFormatFragmenter.
|
Modifier and Type | Method and Description |
---|---|
FragmentsStats |
getFragmentsStats()
Returns statistics for Hive table.
|
static java.lang.String[] |
parseToks(InputData input,
java.lang.String... supportedSerdes)
Extracts the user data:
serde, partition keys and whether filter was included in fragmenter
|
static java.lang.String |
toHiveType(DataType type,
java.lang.String name)
Converts HAWQ type to hive type.
|
getFragments, makeInputFormat
isThreadSafe
public static final int TOK_SERDE
public static final int TOK_KEYS
public static final int TOK_FILTER_DONE
public HiveInputFormatFragmenter(InputData inputData)
inputData
- all input parameters coming from the clientpublic static java.lang.String[] parseToks(InputData input, java.lang.String... supportedSerdes) throws UserDataException
input
- input data from clientsupportedSerdes
- supported serde namesUserDataException
- if user data contains unsupported serde
or wrong number of tokenspublic static java.lang.String toHiveType(DataType type, java.lang.String name)
BOOLEAN -> boolean
SMALLINT -> smallint (tinyint is converted to smallint)
BIGINT -> bigint
TIMESTAMP, TIME -> timestamp
NUMERIC -> decimal
BYTEA -> binary
INTERGER -> int
TEXT -> string
REAL -> float
FLOAT8 -> double
type
- HAWQ data typename
- field nameUnsupportedTypeException
- if type is not supportedpublic FragmentsStats getFragmentsStats() throws java.lang.Exception
getFragmentsStats
in class HiveDataFragmenter
java.lang.Exception
- if statistics cannot be gathered