Automated Editing of Radio Interferometer Data with pieflag
Enno MiddelbergAustralia Telescope National Facility, Epping NSW 1710, Australia. Email: enno.middelberg@csiro.au
Publications of the Astronomical Society of Australia 23(2) 64-68 https://doi.org/10.1071/AS06004
Submitted: 10 January 2006 Accepted: 15 March 2006 Published: 17 May 2006
Abstract
Editing radio interferometer data, a process commonly known as ‘flagging’, can be laborious and time-consuming. One quickly tends to flag more data than actually required, sacrificing sensitivity and image fidelity in the process. I describe a program, pieflag, which can analyze radio interferometer data to filter out measurements which are likely to be affected by interference. pieflag uses two algorithms to allow for data sets which are either dominated by receiver noise or by source structure. Together, the algorithms detect essentially all affected data whilst the amount of data which is not affected by interference but falsely marked as such is kept to a minimum. The sections marked by pieflag are very similar to what would be deemed affected by the observer in a visual inspection of the data. pieflag displays its results concisely and allows the user to add and remove flags interactively. It is written in python, is easy to install and use, and has a variety of options to adjust its algorithms to a particular observing situation. I describe how pieflag works and illustrate its effect using data from typical observations.
Keywords: methods: data analysis
Ellingson,
S. W. 2004, ExA, 17, 261
An error has occurred in (ref id="R1").
Variable LPAGE is undefined. [empty string]
array
1
struct
COLUMN
0
ID
??
LINE
243
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor10(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:243)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
2
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
180
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor12(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:180)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
3
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
177
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor13(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:177)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
4
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
176
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor14(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:176)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
5
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
170
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor40(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:170)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
6
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
15
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor41(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:15)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
7
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
12
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor42(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:12)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
8
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
11
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986._factor43(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:11)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
9
struct
COLUMN
0
ID
CF_INCLUDEJOURNALREF_B
LINE
1
RAW_TRACE
at cfincludeJournalRef_B2ecfm1510229986.runPage(G:\www\publishha\app\views\journal\includeJournalRef_B.cfm:1)
TEMPLATE
G:\www\publishha\app\views\journal\includeJournalRef_B.cfm
TYPE
CFML
10
struct
COLUMN
0
ID
CFINCLUDE
LINE
3230
RAW_TRACE
at cfArticle2ecfc1291681887$funcREFERENCESDTD1._factor150(G:\www\publishha\app\modules\journal\Article.cfc:3230)
TEMPLATE
G:\www\publishha\app\modules\journal\Article.cfc
TYPE
CFML
11
struct
COLUMN
0
ID
CF_ARTICLE
LINE
3217
RAW_TRACE
at cfArticle2ecfc1291681887$funcREFERENCESDTD1._factor151(G:\www\publishha\app\modules\journal\Article.cfc:3217)
TEMPLATE
G:\www\publishha\app\modules\journal\Article.cfc
TYPE
CFML
12
struct
COLUMN
0
ID
CF_ARTICLE
LINE
3216
RAW_TRACE
at cfArticle2ecfc1291681887$funcREFERENCESDTD1.runFunction(G:\www\publishha\app\modules\journal\Article.cfc:3216)
TEMPLATE
G:\www\publishha\app\modules\journal\Article.cfc
TYPE
CFML
13
struct
COLUMN
0
ID
CF_TEMPLATEPROXY
LINE
2978
RAW_TRACE
at cfArticle2ecfc1291681887$funcBACKSECTIONSFORABSTRACTONLYDTD1.runFunction(G:\www\publishha\app\modules\journal\Article.cfc:2978)
TEMPLATE
G:\www\publishha\app\modules\journal\Article.cfc
TYPE
CFML
14
struct
COLUMN
0
ID
CF_TEMPLATEPROXY
LINE
118
RAW_TRACE
at cfgenArticleContent2ecfm782833447._factor2(G:\www\publishha\app\views\journal\generator\genArticleContent.cfm:118)
TEMPLATE
G:\www\publishha\app\views\journal\generator\genArticleContent.cfm
TYPE
CFML
15
struct
COLUMN
0
ID
CF_GENARTICLECONTENT
LINE
99
RAW_TRACE
at cfgenArticleContent2ecfm782833447._factor4(G:\www\publishha\app\views\journal\generator\genArticleContent.cfm:99)
TEMPLATE
G:\www\publishha\app\views\journal\generator\genArticleContent.cfm
TYPE
CFML
16
struct
COLUMN
0
ID
CF_GENARTICLECONTENT
LINE
1
RAW_TRACE
at cfgenArticleContent2ecfm782833447._factor6(G:\www\publishha\app\views\journal\generator\genArticleContent.cfm:1)
TEMPLATE
G:\www\publishha\app\views\journal\generator\genArticleContent.cfm
TYPE
CFML
17
struct
COLUMN
0
ID
CF_GENARTICLECONTENT
LINE
1
RAW_TRACE
at cfgenArticleContent2ecfm782833447.runPage(G:\www\publishha\app\views\journal\generator\genArticleContent.cfm:1)
TEMPLATE
G:\www\publishha\app\views\journal\generator\genArticleContent.cfm
TYPE
CFML
18
struct
COLUMN
0
ID
CFINCLUDE
LINE
10
RAW_TRACE
at cfStaticPageLayout2ecfm1426351739.runPage(G:\www\publishha\app\views\layout\StaticPageLayout.cfm:10)
TEMPLATE
G:\www\publishha\app\views\layout\StaticPageLayout.cfm
TYPE
CFML
19
struct
COLUMN
0
ID
CFINCLUDE
LINE
30
RAW_TRACE
at cfCacheController2ecfc639388092$funcRENDERLAYOUT.runFunction(G:\www\publishha\app\controller\CacheController.cfc:30)
TEMPLATE
G:\www\publishha\app\controller\CacheController.cfc
TYPE
CFML
20
struct
COLUMN
0
ID
CF_UDFMETHOD
LINE
241
RAW_TRACE
at cfCacheController2ecfc639388092$funcGENARTICLECONTENT.runFunction(G:\www\publishha\app\controller\CacheController.cfc:241)
TEMPLATE
G:\www\publishha\app\controller\CacheController.cfc
TYPE
CFML
21
struct
COLUMN
0
ID
CF_UDFMETHOD
LINE
327
RAW_TRACE
at cfCacheController2ecfc639388092$funcGENARTICLE.runFunction(G:\www\publishha\app\controller\CacheController.cfc:327)
TEMPLATE
G:\www\publishha\app\controller\CacheController.cfc
TYPE
CFML
22
struct
COLUMN
0
ID
CF_UDFMETHOD
LINE
39
RAW_TRACE
at cfCacheController2ecfc639388092$funcGENCACHE.runFunction(G:\www\publishha\app\controller\CacheController.cfc:39)
TEMPLATE
G:\www\publishha\app\controller\CacheController.cfc
TYPE
CFML
23
struct
COLUMN
0
ID
CF_CFPAGE
LINE
634
RAW_TRACE
at cfPublishingApp2ecfc536529069$funcPROCESSREQUEST.runFunction(G:\www\publishha\app\PublishingApp.cfc:634)
TEMPLATE
G:\www\publishha\app\PublishingApp.cfc
TYPE
CFML
24
struct
COLUMN
0
ID
CF_TEMPLATEPROXY
LINE
16
RAW_TRACE
at cfBootstrap2ecfc660975501$funcHANDLEREQUEST.runFunction(G:\www\publishha\app\Bootstrap.cfc:16)
TEMPLATE
G:\www\publishha\app\Bootstrap.cfc
TYPE
CFML
25
struct
COLUMN
0
ID
CF_TEMPLATEPROXY
LINE
18
RAW_TRACE
at cfApplication2ecfc757760907$funcONREQUESTSTART.runFunction(G:\www\publishha\webroot\Application.cfc:18)
TEMPLATE
G:\www\publishha\webroot\Application.cfc
TYPE
CFML
1 www.astron.nl/~renting/flagger.html
2 The original name of the program, ‘pyflag’, turned out to be already in use, requiring a name modification of some sort to avoid confusion.
3 www.narrabri.atnf.csiro.au/observing/rfi/
4 www.atnf.csiro.au/people/Enno.Middelberg/pieflag