首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >Awk/Perl将文本文件转换为csv格式

Awk/Perl将文本文件转换为csv格式
EN

Stack Overflow用户
提问于 2013-04-10 18:00:26
回答 3查看 1.5K关注 0票数 2

我有一个具有以下格式的历史自动生成的日志文件,我希望在上载到数据库之前将其转换为csv文件

代码语言:javascript
复制
--------------------------------------
Thu Jul  8 09:34:12 BST 2010
BLUE Head 1
Duration = 20 s
Activity = 14.9 MBq
Sensitivity = 312 cps/MBq
--------------------------------------
Thu Jul  8 09:34:55 BST 2010
BLUE Head 1
Duration = 20 s
Activity = 14.9 MBq
Sensitivity = 318 cps/MBq
--------------------------------------
Thu Jul  8 10:13:39 BST 2010
RED Head 1
Duration = 20 s
Activity = 14.9 MBq
Sensitivity = 307 cps/MBq
--------------------------------------
Thu Jul  8 10:14:10 BST 2010
RED Head 1
Duration = 20 s
Activity = 14.9 MBq
Sensitivity = 305 cps/MBq
--------------------------------------
Mon Jul 19 10:11:18 BST 2010
BLUE Head 1
Duration = 20 s
Activity = 12.4 MBq
Sensitivity = 326 cps/MBq
--------------------------------------
Mon Jul 19 10:12:09 BST 2010
BLUE Head 1
Duration = 20 s
Activity = 12.4 MBq
Sensitivity = 333 cps/MBq
--------------------------------------
Mon Jul 19 10:13:57 BST 2010
RED Head 1
Duration = 20 s
Activity = 12.4 MBq
Sensitivity = 338 cps/MBq
--------------------------------------
Mon Jul 19 10:14:45 BST 2010
RED Head 1
Duration = 20 s
Activity = 12.4 MBq
Sensitivity = 340 cps/MBq
--------------------------------------

我想将日志文件转换为以下格式

代码语言:javascript
复制
Date,Camera,Head,Duration,Activity
08/07/10,BLUE,1,20,14.9
08/07/10,BLUE,1,20,14.9
08/07/10,RED,1,20,14.9
08/07/10,RED,1,20,14.9

我使用awk让我接近我想要的东西

代码语言:javascript
复制
awk 'BEGIN {print "Date,Camera,Head,Duration,Activity";RS = "--------------------------------------"; FS="\n";}; {OFS=",";split($3, a, " ");split($4,b, " "); split($5,c," ");print $2,a[1],a[3],b[3],c[3]}' sensitivity.txt > sensitivity.csv

这给了我

代码语言:javascript
复制
Date,Camera,Head,Duration,Activity
,,,,
Thu Jul  8 09:34:12 BST 2010,BLUE,1,20,14.9
Thu Jul  8 09:34:55 BST 2010,BLUE,1,20,14.9
Thu Jul  8 10:13:39 BST 2010,RED,1,20,14.9
Thu Jul  8 10:14:10 BST 2010,RED,1,20,14.9

我怎样才能

(a)去掉第4行中的4个输出字段分隔符(b)将日期格式从清华Jul 8 09:34:12 BST 2010转换为DD/MM/YY (我可以使用纯awk或通过管道将其转换为perl)

EN

回答 3

Stack Overflow用户

回答已采纳

发布于 2013-04-10 18:41:05

这个简单的awk脚本将完成这项工作:

代码语言:javascript
复制
BEGIN {
    n=split("Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec",month,"|")
    for (i=1;i<=n;i++) {
        month_index[month[i]] = i
    }
    print "Date,Camera,Head,Duration,Activity"  
}
/^-*$/{
    i=0
    next
}
{
    i++
}
i==1{
    printf "%02d/%02d/%02d,",$3,month_index[$2],substr($6,3)
}
i==2{
    printf "%s,%d,",$1,$3
}
i==3{
    printf "%d,",$3
}
i==4{
    printf "%.1f\n",$3
}

输出:

代码语言:javascript
复制
$ awk -f script.awk file
08/07/10,BLUE,1,20,14.9
08/07/10,BLUE,1,20,14.9
08/07/10,RED,1,20,14.9
08/07/10,RED,1,20,14.9
19/07/10,BLUE,1,20,12.4
19/07/10,BLUE,1,20,12.4
19/07/10,RED,1,20,12.4
19/07/10,RED,1,20,12.4
票数 1
EN

Stack Overflow用户

发布于 2013-04-10 21:11:40

@sudo_O的答案很好,但这里有一个替代方案:

代码语言:javascript
复制
$ cat tst.awk
BEGIN{ RS="---+\n"; OFS=","; months="JanFebMarAprMayJunJulAugSepOctNovDec" }
NR==1{ print "Date","Camera","Head","Duration","Activity"; next }
{ print sprintf("%04d%02d%02d",$6,(match(months,$2)+2)/3,$3),$7,$9,$12,$16 }

$ gawk -f tst.awk file
Date,Camera,Head,Duration,Activity
20100708,BLUE,1,20,14.9
20100708,BLUE,1,20,14.9
20100708,RED,1,20,14.9
20100708,RED,1,20,14.9
20100719,BLUE,1,20,12.4
20100719,BLUE,1,20,12.4
20100719,RED,1,20,12.4
20100719,RED,1,20,12.4

注意,我使用了上面的GNU awk,所以我可以将RS设置为多个字符。对于其他awk,只需将所有“-...”行转换为空行或控制字符或其他东西,并在运行脚本之前相应地设置RS即可。

如果您不喜欢我建议的日期格式,可以调整sprintf()以适合它。

票数 2
EN

Stack Overflow用户

发布于 2013-04-11 12:01:12

我想我应该展示如何实际解析输入,而不仅仅是执行字符串转换。

代码语言:javascript
复制
#! /usr/bin/env perl
use strict;
use warnings;
use Date::Parse;
use Date::Format;
use Text::CSV;

sub convert_date{
  my $time = str2time($_[0]);
  # iso 8601 style:
  return time2str('%Y-%m-%d',$time); # YYYY-MM-DD

  # or the outdated style output you wanted
  return time2str('%d/%m/%y',$time); # DD/MM/YY
}

my %multiply_table = (
  s => 1,
  m => 60,
  h => 60 * 60,
  d => 60 * 60 * 24,
);
sub convert_duration{
  my($d,$s) = $_[0] =~ /^ \s* (\d+) \s* (\w) \s* $/x;
  die "Invalid duration '$_[0]'" unless $d && $s;
  return $d * $multiply_table{$s};
}

my @field_list = qw'Date Camera Head Duration Activity';

my $csv = Text::CSV->new( { eol => "\n" } );

# print header
$csv->print( \*STDOUT, \@field_list );

# set record separator
local $/ = ('-' x 38) . "\n";

# parse data
while(<>){
  chomp; # remove record separator
  next unless $_; # skip empty section
  my($time,$camdat,@fields) = split m/\n/; # split up the fields

  my %data;


  # split camera and head fields
  @data{qw(Camera Head)} = split /\s+Head\s+/, $camdat;

  # parse lines like:
  #   Duration = 20 s
  #   Activity = 14.9 MBq
  #   Sensitivity = 305 cps/MBq
  for(@fields){
    my($key,$value) = /(\w+) \s* = \s* (.*) /x;
    $data{$key} = $value;
  }

  # at this point we start reducing precision

  $data{Date} = convert_date( $time );

  # remove measurement units
  $data{Duration} = convert_duration($data{Duration}); # safe
  $data{Activity} =~ s/[^\d]*$//; # unsafe

  $csv->print(\*STDOUT, [@data{@field_list}]);
}
票数 1
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/15922706

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档