Pig dump 运算符



  • 诊断运算符

    load 语句将简单地将数据加载到Apache pig 指定的关系。要验证Load语句的执行,您必须使用Diagnostic(诊断)操作符。Pig Latin提供四种不同类型的诊断运算符-
    • dump 运算符
    • Describe 运算符
    • Explanation 运算符
    • Illustration 运算符
    在本章中,我们将讨论Pig Latin的Dump运算符。
  • dump 运算符

    dump 操作来运行的Pig Latin语句并在屏幕上显示的结果。它通常用于调试目的。
    句法
    下面给出的是Dump运算符的语法。
    
    grunt> Dump Relation_Name
    
    
    例子
    假设我们在HDFS中有一个具有以下内容的文件Student_data.txt。
    
    001,Rajiv,Reddy,9848022337,Hyderabad
    002,siddarth,Battacharya,9848022338,Kolkata
    003,Rajesh,Khanna,9848022339,Delhi
    004,Preethi,Agarwal,9848022330,Pune
    005,Trupthi,Mohanthy,9848022336,Bhuwaneshwar
    006,Archana,Mishra,9848022335,Chennai.
    
    
    如下所示,我们已使用LOAD运算符将其读入关系学生。
    
    grunt> student = LOAD 'hdfs://localhost:9000/pig_data/student_data.txt' 
       USING PigStorage(',')
       as ( id:int, firstname:chararray, lastname:chararray, phone:chararray,city:chararray );
    
    
    现在,让我们使用Dump运算符打印关系的内容,如下所示。
    
    grunt> Dump student
    
    
    一旦执行了上述Pig Latin语句,它将启动MapReduce作业以从HDFS读取数据。它将产生以下输出。
    
    2020-10-01 15:05:27,642 [main]
    INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 
    100% complete
    2020-10-01 15:05:27,652 [main]
    INFO  org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics:   
    HadoopVersion  PigVersion  UserId    StartedAt             FinishedAt       Features             
    2.6.0          0.15.0      Hadoop  2020-10-01 15:03:11  2020-10-01 05:27     UNKNOWN
                                                    
    Success!  
    Job Stats (time in seconds):
      
    JobId           job_14459_0004
    Maps                 1  
    Reduces              0  
    MaxMapTime          n/a    
    MinMapTime          n/a
    AvgMapTime          n/a 
    MedianMapTime       n/a
    MaxReduceTime        0
    MinReduceTime        0  
    AvgReduceTime        0
    MedianReducetime     0
    Alias             student 
    Feature           MAP_ONLY        
    Outputs           hdfs://localhost:9000/tmp/temp580182027/tmp757878456,
    
    Input(s): Successfully read 0 records from: "hdfs://localhost:9000/pig_data/
    student_data.txt"
      
    Output(s): Successfully stored 0 records in: "hdfs://localhost:9000/tmp/temp580182027/
    tmp757878456"  
    
    Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager 
    spill count : 0Total bags proactively spilled: 0 Total records proactively spilled: 0  
    
    Job DAG: job_1443519499159_0004
      
    2020-10-01 15:06:28,403 [main]
    INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLau ncher - Success!
    2020-10-01 15:06:28,441 [main] INFO  org.apache.pig.data.SchemaTupleBackend - 
    Key [pig.schematuple] was not set... will not generate code.
    2020-10-01 15:06:28,485 [main]
    INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths 
    to process : 1
    2020-10-01 15:06:28,485 [main]
    INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths
    to process : 1
    
    (1,Rajiv,Reddy,9848022337,Hyderabad)
    (2,siddarth,Battacharya,9848022338,Kolkata)
    (3,Rajesh,Khanna,9848022339,Delhi)
    (4,Preethi,Agarwal,9848022330,Pune)
    (5,Trupthi,Mohanthy,9848022336,Bhuwaneshwar)
    (6,Archana,Mishra,9848022335,Chennai)