Package org.apache.beam.sdk.extensions.jackson

Class ParseJsons.ParseJsonsWithFailures<FailureT>

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    ParseJsons<OutputT>
    public class ParseJsons.ParseJsonsWithFailures<FailureT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<java.lang.String>,​org.apache.beam.sdk.transforms.WithFailures.Result<org.apache.beam.sdk.values.PCollection<OutputT>,​FailureT>>
    A PTransform that adds exception handling to ParseJsons.
    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.transforms.PTransform

        annotations, displayData, name, resourceHints

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      ParseJsons.ParseJsonsWithFailures<FailureT> exceptionsVia​(org.apache.beam.sdk.transforms.ProcessFunction<org.apache.beam.sdk.transforms.WithFailures.ExceptionElement<java.lang.String>,​FailureT> exceptionHandler)
      Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, passing the raised exception instance and the input element being processed through the given exceptionHandler and emitting the result to a failure collection.
      org.apache.beam.sdk.transforms.WithFailures.Result<org.apache.beam.sdk.values.PCollection<OutputT>,​FailureT> expand​(org.apache.beam.sdk.values.PCollection<java.lang.String> input)  

      • Methods inherited from class org.apache.beam.sdk.transforms.PTransform

        addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Method Detail

      • exceptionsVia

        public ParseJsons.ParseJsonsWithFailures<FailureT> exceptionsVia​(org.apache.beam.sdk.transforms.ProcessFunction<org.apache.beam.sdk.transforms.WithFailures.ExceptionElement<java.lang.String>,​FailureT> exceptionHandler)
        Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, passing the raised exception instance and the input element being processed through the given exceptionHandler and emitting the result to a failure collection. It is supposed to be used along with ParseJsons.exceptionsInto(TypeDescriptor) and get lambda function as exception handler.

        See WithFailures documentation for usage patterns of the returned WithFailures.Result.

        Example usage: 

        
 WithFailures.Result<PCollection<MyPojo>, KV<String, Map<String, String>>> result =
     json.apply(
         ParseJsons.of(MyPojo.class)
             .exceptionsInto(
                 TypeDescriptors.kvs(TypeDescriptors.strings(), TypeDescriptors.strings()))
             .exceptionsVia(
                 f -> KV.of(f.element(), f.exception().getClass().getCanonicalName())));

 PCollection<MyPojo> output = result.output(); // valid POJOs
 PCollection<KV<String, Map<String, String>>> failures = result.failures();

      • expand

        public org.apache.beam.sdk.transforms.WithFailures.Result<org.apache.beam.sdk.values.PCollection<OutputT>,​FailureT> expand​(org.apache.beam.sdk.values.PCollection<java.lang.String> input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<java.lang.String>,​org.apache.beam.sdk.transforms.WithFailures.Result<org.apache.beam.sdk.values.PCollection<OutputT>,​FailureT>>