uk.ac.ebi.arrayexpress2.magetab.utils

Class MAGETABUtils

java.lang.Object

uk.ac.ebi.arrayexpress2.magetab.utils.MAGETABUtils

public class MAGETABUtils
extends Object

Some common utilities for working with MAGE-TAB files.

Author:

Tony Burdett

Constructor Summary

Constructors

Constructor and Description
MAGETABUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static String	compensateForEscapedNewlines(String firstLine, String secondLine) Deprecated.
static String	digestHeader(String header) Remove any unescaped or unparenthesised whitespace, and convert to lower case for a single header field
static String[]	digestHeaders(String[] header) Remove any unescaped or unparenthesised whitespace, and convert to lower case for a row of headers
static boolean	endsWithEscapedNewline(String line) Deprecated.
static String	escapeCell(String cell) Performs all necessary escaping for cell values using the following rules: - all TAB characters replaced with spaces - cells containing CR and/or LF characters are quoted using double-quotes - existing quoting is preserved - all double-quote characters inside cell will be escaped by pre-pending backslash - existing escaping is preserved
static <T> T[]	extractRange(T[] array, int startIndex) Extract a subset of an array from a larger array.
static <T> T[]	extractRange(T[] array, int startIndex, int endIndex) Extract a subset of an array from a larger array.
static String	extractSubtype(String header) Extract the subtype given in the header.
static String	extractType(String header) Extract the type given in the header.
static String[][]	insertColumn(String[][] data, String header, int insertBefore)
static boolean	isAnchoredAttribute(String parentHeader, String attributeHeader) Determines whether the supplied attribute header is anchored to the parent header.
static boolean	isMergedTabDelimitedInputStream(InputStream in, String encoding, String... sectionTags) If possible, checks whether the supplied input stream reads from a merged MAGE-TAB file.
static File	locateIDF(File dir) Given a directory that represents a MAGE-TAB document, locate the IDF file it contains
static File	locateSDRF(File dir) Given a directory that represents a MAGE-TAB document, locate the SDRF file it contains
static String[][]	readMergedTabDelimitedInputStream(InputStream in, String encoding, boolean stripEscaping, boolean trimWhitespace, String... sectionTags) Deprecated.
static String[][]	readMergedTabDelimitedInputStream(InputStream in, String encoding, boolean stripEscaping, String... sectionTags) Deprecated.
static String[][]	readTabDelimitedInputStream(InputStream in, String encoding, boolean stripEscaping) Deprecated.
static String[][]	readTabDelimitedInputStream(InputStream in, String encoding, boolean stripEscaping, boolean trimWhitespace) Deprecated.
static String[]	removeEscaping(String line) Deprecated.
static String[]	removeEscaping(String[] cells) Deprecated.
static String[]	removeEscaping(String[] cells, boolean useStrict) Deprecated.
static String[]	removeEscaping(String line, boolean useStrict) Deprecated.
static String[]	removeTrailingEmptyCells(String[] row) Deprecated.
static String[]	removeWhitespace(String[] cells) Deprecated.
static Map<String,String>	resolveExperimentFactorTypes(MAGETABInvestigation investigation, HybridizationNode hybNode) Resolves all the factor value types (the typing information in square brackets - so "foo" in Factor Value[foo]) in the provided hybridization node and maps them to the actual experiment factor type declared in the IDF.
static URL	resolveRelativeLocation(URL currentLocation, String relativeFileName) Convenience method for obtaining the URL for a file given only a relative path and the URL of the document the relative path is declared in.
static String[]	splitLine(String line) Deprecated.
static String[]	splitLine(String line, boolean ignoreEscaping) Deprecated.
static String[]	splitLine(String line, boolean ignoreEscaping, boolean trimWhitespace) Deprecated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MAGETABUtils

public MAGETABUtils()

Method Detail

readTabDelimitedInputStream

@Deprecated
public static String[][] readTabDelimitedInputStream(InputStream in,
                                                String encoding,
                                                boolean stripEscaping,
                                                boolean trimWhitespace)
                                              throws IOException

Deprecated.

Reads data from an input stream into a two dimensional string array, organised by row then column. Iterating over the result of this operation gives an array of string arrays, where each element in the inner array represents on column of the line.

Parameters:

in - the input stream to read from

encoding - the encoding type of the input stream

stripEscaping - whether to remove any quotes used to escape cells before returning the result

trimWhitespace - whether to remove whitespace from lines before splitting into cells

Returns:

an array of string[], where each string[] represents one line read from the input stream

Throws:

IOException - if the stream could not be read

readTabDelimitedInputStream

@Deprecated
public static String[][] readTabDelimitedInputStream(InputStream in,
                                                String encoding,
                                                boolean stripEscaping)
                                              throws IOException

Deprecated.

Reads data from an input stream into a two dimensional string array, organised by row then column. Iterating over the result of this operation gives an array of string arrays, where each element in the inner array represents on column of the line. By default this will trim whitespace from lines before splitting into cells.

Parameters:

in - the input stream to read from

encoding - the encoding type of the input stream

stripEscaping - whether to remove any quotes used to escape cells before returning the result

Returns:

an array of string[], where each string[] represents one line read from the input stream

Throws:

IOException - if the stream could not be read

readMergedTabDelimitedInputStream

@Deprecated
public static String[][] readMergedTabDelimitedInputStream(InputStream in,
                                                      String encoding,
                                                      boolean stripEscaping,
                                                      String... sectionTags)
                                                    throws IOException

Deprecated.

Reads data from an input stream into a two dimensional string array, organised by row then column. Iterating over the result of this operation gives an array of string arrays, where each element in the inner array represents on column of the line.

This method differs from the alternate form readTabDelimitedInputStream(java.io.InputStream, String, boolean) in that it is capable of reading a file divided into sections by tags that indicate the split. In the case of some MAGE-TAB files, this is denoted by [IDF] and [SDRF] tags. Such tags should be passed as the sectionTags parameter. This method skips the first section tag if it occurs before other content, and then breaks at any subsequent section markers it encounters.

Parameters:

in - the input stream to read from

encoding - the encoding type of the input stream

stripEscaping - whether to remove any quotes used to escape cells before returning the result

sectionTags - the labels that delineate sections in this stream (one per line)

Returns:

an array of string[], where each string[] represents one line read from the input stream

Throws:

IOException - if the stream could not be read

readMergedTabDelimitedInputStream

@Deprecated
public static String[][] readMergedTabDelimitedInputStream(InputStream in,
                                                      String encoding,
                                                      boolean stripEscaping,
                                                      boolean trimWhitespace,
                                                      String... sectionTags)
                                                    throws IOException

Deprecated.

Reads data from an input stream into a two dimensional string array, organised by row then column. Iterating over the result of this operation gives an array of string arrays, where each element in the inner array represents on column of the line.

This method differs from the alternate form readTabDelimitedInputStream(java.io.InputStream, String, boolean) in that it is capable of reading a file divided into sections by tags that indicate the split. In the case of some MAGE-TAB files, this is denoted by [IDF] and [SDRF] tags. Such tags should be passed as the sectionTags parameter. This method skips the first section tag if it occurs before other content, and then breaks at any subsequent section markers it encounters.

Parameters:

in - the input stream to read from

encoding - the encoding type of the input stream

stripEscaping - whether to remove any quotes used to escape cells before returning the result

trimWhitespace - whether to remove whitespace from lines before splitting into cells

sectionTags - the labels that delineate sections in this stream (one per line)

Returns:

an array of string[], where each string[] represents one line read from the input stream

Throws:

IOException - if the stream could not be read

isMergedTabDelimitedInputStream

public static boolean isMergedTabDelimitedInputStream(InputStream in,
                                      String encoding,
                                      String... sectionTags)
                                               throws IOException

If possible, checks whether the supplied input stream reads from a merged MAGE-TAB file. This check is performed by testing for the presence of any of the sectionTags supplied on the first line with content. If such a tag is found, this method returns true. If not, it returns false. Note that in order to perform this check, the input stream is marked and read, then reset. If the supplied input stream does not support marking operations, an IOException is thrown

Parameters:

in - the input stream to test

encoding - the encoding type of the input stream

sectionTags - the tags that

Returns:

true if the given stream represents a merged MAGE-TAB document, false otherwise

Throws:

IOException - if the underlying stream does not support the mark() operation

insertColumn

public static String[][] insertColumn(String[][] data,
                      String header,
                      int insertBefore)

removeTrailingEmptyCells

@Deprecated
public static String[] removeTrailingEmptyCells(String[] row)

Deprecated.

Take a row of cells and removes any trailing cells that contain empty Strings

Parameters:

row - the cells to remove trailing whitespace from

Returns:

a shortened string array with any trailing empty cells removed

endsWithEscapedNewline

@Deprecated
public static boolean endsWithEscapedNewline(String line)

Deprecated.

Determine whether the string supplied ends with an escaped newline character. If the line does not end with a newline character, it is assumed to have been previously removed, and therefore this will determine if the line ends "within" a quote.

Parameters:

line - the line to inspect

Returns:

true if the string ends "within" a quote, false otherwise

compensateForEscapedNewlines

@Deprecated
public static String compensateForEscapedNewlines(String firstLine,
                                             String secondLine)

Deprecated.

Compensate for a newline character that has been inserted into a single 'logical' line in a MAGETAB file and then escaped. This form of escaping is legal in the MAGETAB specification, but obviously extremely unhelpful for automated parsing, as one "logical" line in the file could potentially be spread across several actual lines in the file, with quotes around the newline character meant to indicate escaping. This method takes two lines, which have been judged to be separated by an escaped newline character (see endsWithEscapedNewline(String)) and then concatenates the two strings. Note that it will first check whether the first line does indeed end in an escaped newline, and if not it will throw an exception. However, there are situations when this will fail. When parsing a single line, whether this line ends in an escaped newline character is judged by checking each tab-delimited cell and if there are cells that start with a quote and do NOT end in a quote before the end of the line, this line is assumed to end in an escaped newline character. If there are consecutive lines ending in this way, however, every second line will look like this yet will NOT end with an escaped newline. Therefore, care must be taken to remove all escaped newline characters in order, concatenating each string as you parse, before calling this method.

Parameters:

firstLine - the first line, i.e. the one ending with an escaped newline character

secondLine - the second line, i.e. the one that is a logical continuation of the first

Returns:

the result string, with the newline character removed

splitLine

@Deprecated
public static String[] splitLine(String line)

Deprecated.

Splits this line into appropriate cells in accordance with the MAGE-TAB specification. Empty cells are preserved as empty strings. This is equivalent to calling splitLine(line, false); (see splitLine(String, boolean)).

Parameters:

line - the line to split

Returns:

the resulting string array

splitLine

@Deprecated
public static String[] splitLine(String line,
                            boolean ignoreEscaping)

Deprecated.

Splits this line into appropriate cells in accordance with the MAGE-TAB specification. If escaping is present around any strings on this line, the "logical" cells will be different from the "physical" cells - any quoted tab characters will not be considered to be cell delimiters (i.e. some cells may contain tab characters) - unless ignoreEscaping is set to true. Empty cells are preserved as empty strings. This is equivalent to calling splitLine(line, ignoreEscaping, true); (see splitLine(String, boolean, boolean)).

Parameters:

line - the line to split

ignoreEscaping - whether to ignore escaped tab and newline characters or whether to handle them according to the MAGE-TAB spec.

Returns:

the resulting string array

splitLine

@Deprecated
public static String[] splitLine(String line,
                            boolean ignoreEscaping,
                            boolean trimWhitespace)

Deprecated.

Splits this line into appropriate cells in accordance with the MAGE-TAB specification, ingoring any escaping found. Empty cells are preserved as empty strings. Any trailing whitespace in cells present is trimmed if 'trimWhitespace' is true, otherwise every cell is preserved verbosely.

Parameters:

line - the line to split

ignoreEscaping - whether to ignore escaped tab and newline characters or whether to handle them according to the MAGE-TAB spec.

trimWhitespace - whether to act in 'strict' mode, preserving any leading and trailing whitespace characters, or whether to automatically remove them

Returns:

the resulting string array

removeEscaping

@Deprecated
public static String[] removeEscaping(String line)

Deprecated.

Convenience method to split and remove all escaping from a line of MAGE-TAB data. Equivalent to calling splitLine(line, false); followed by removeEscaping(cells); on the result.

Parameters:

line - the line containing cells to remove escaping from

Returns:

the resulting logical cells after the line has been split and escaping removed

removeEscaping

@Deprecated
public static String[] removeEscaping(String line,
                                 boolean useStrict)

Deprecated.

Convenience method to split and remove all escaping from a line of MAGE-TAB data. Equivalent to calling splitLine(line); followed by removeEscaping(cells); on the result.

Parameters:

line - the line containing cells to remove escaping from

useStrict - whether to strictly remove all quotation marks from this cell, or whether to consider only those at the beginning and end of each cell.

Returns:

the resulting logical cells after the line has been split and escaping removed

removeEscaping

@Deprecated
public static String[] removeEscaping(String[] cells)

Deprecated.

Removes any quotation marks around the given cells that are used to escape the values contained within, using relaxed mode by default. Equivalent to removeEscaping(cells, false);

Parameters:

cells - the cells to remove escaping from

Returns:

the resulting cells, after escaping has been removed

removeEscaping

@Deprecated
public static String[] removeEscaping(String[] cells,
                                 boolean useStrict)

Deprecated.

Removes any escaping present in the given cells using either a strict or relaxed interpretation of the MAGE-TAB specification..

In MAGETAB quotation marks are used to surround some values, and this signifies that they should be used as part of the value supplied; this includes tabs, newlines and other such characters.

In 'strict' mode, all such quotation marks are removed from the whole cell. Strict checking requires regular expression checking, and therefore can fail if the string contains irregular markup (for example, HTML fragments). Also note that this method will not work with lines that end in an escaped newline character, and you should explicitly remove these characters first using the endsWithEscapedNewline(String) and compensateForEscapedNewlines(String, String) methods first.

In 'relaxed' mode, this method removes quotations ONLY when present at the beginning and end of a cell. This form is different from strict checking in that it does not use regular expressions to check for quoted substrings - rather, it just checks the first and last character of each cell for quotations. Relaxed mode will work with lines that end in an escaped newline character or may not pass regular expression checks.

Generally, MAGE-TAB convention is to only quote around cells (and not within cells) to imply escaping, so relaxed mode is almost always the preferred option. For thorough sensitivity use the strict form,

Parameters:

cells - the logical cells on a line of MAGE-TAB that may contain segments escaped with quotation marks

useStrict - whether to use the strict definition of escaping from the MAGE-TAB spec, or whether to consider quotations at the beginning and end of cells only.

Returns:

a cell with all quotations removed

removeWhitespace

@Deprecated
public static String[] removeWhitespace(String[] cells)

Deprecated.

Removes any whitespace from the beginning or the end of a cell. This is present to ensure that unique identifiers with arbitrary whitespace surrounding them resolve to the same object (so, for example, Protocol "P-ABCD-1234" and "P-ABCD-1234 " identify the same protocol).

Parameters:

cells - the cells to trim leading and trailing whitespace from

Returns:

the resulting cells, after escaping has been removed

digestHeader

public static String digestHeader(String header)

Remove any unescaped or unparenthesised whitespace, and convert to lower case for a single header field

Parameters:

header - the header to digest

Returns:

the digested string

digestHeaders

public static String[] digestHeaders(String[] header)

Remove any unescaped or unparenthesised whitespace, and convert to lower case for a row of headers

Parameters:

header - the header to digest

Returns:

the digested string

extractType

public static String extractType(String header)

Extract the type given in the header. In other words, return any values in [...].

Parameters:

header - the header to read

Returns:

the type value, if any

extractSubtype

public static String extractSubtype(String header)

Extract the subtype given in the header. In other words, return any values in (...).

Note that the MAGE-TAB spec does not explicitly forbid the use of parentheses inside square brackets. This means, for example, that an SDRF could contain a typing like "Factor Value[Growth (foo) Condition]". In this case, the (foo) part is not a subtype, instead it is part of the typing. Therefore this method throws away anything insde square bracktes before checking for parentheses.

Parameters:

header - the header to read

Returns:

the type value, if any

resolveRelativeLocation

public static URL resolveRelativeLocation(URL currentLocation,
                          String relativeFileName)
                                   throws IOException

Convenience method for obtaining the URL for a file given only a relative path and the URL of the document the relative path is declared in. This should work on windows-based machines as well as linux and MAC ones.

Parameters:

currentLocation - the location of the declaring document

relativeFileName - the path to the declared file relative to the declaring document

Returns:

the URL to the declared file

Throws:

IOException - if the path declared does not declare a valid path

locateIDF

public static File locateIDF(File dir)

Given a directory that represents a MAGE-TAB document, locate the IDF file it contains

Parameters:

dir - the directory containing all the MAGE-TAB documents

Returns:

the IDF file, or null if none was found

locateSDRF

public static File locateSDRF(File dir)

Given a directory that represents a MAGE-TAB document, locate the SDRF file it contains

Parameters:

dir - the directory containing all the MAGE-TAB documents

Returns:

the SDRF file, or null if none was found

extractRange

public static <T> T[] extractRange(T[] array,
                   int startIndex,
                   int endIndex)

Extract a subset of an array from a larger array. Elements from startIndex to endIndex inclusive will be extracted and returned as a new array. The type of the array will be preserved. The start and index values are zero indexed and are inclusive: in other words, the maximum range that can be extracted is from 0 to array.length-1. If the endIndex value is greater than this, the result would be the same as using the extractRange(Object[], int) form of this method - it will extract to the end and no further.

Type Parameters:

T - the type of the array

Parameters:

array - the array to extract a subset from

startIndex - the first element to extract, zero indexed, inclusive

endIndex - the last element to extract, zero indexed, inclusive - if this is greater than the length of the array this will extract to the end of the array

Returns:

an array comprising a subset of the elements in the original array

extractRange

public static <T> T[] extractRange(T[] array,
                   int startIndex)

Extract a subset of an array from a larger array. Elements from startIndex to the end of the array, inclusive will be extracted and returned as a new array. The type of the array will be preserved.

Type Parameters:

T - the type of the array

Parameters:

array - the array to extract a subset from

startIndex - the first element to extract, zero indexed, inclusive

Returns:

an array comprising a subset of the elements in the original array

isAnchoredAttribute

public static boolean isAnchoredAttribute(String parentHeader,
                          String attributeHeader)

Determines whether the supplied attribute header is anchored to the parent header. An anchored attribute is one which cannot be separated from the attribute it is associated with. This is normally only applicable to Term Source REF, Term Accession Number, and Unit columns, but Comments can be anchored to attributes in some cases (provider, performer and parameter values)

Parameters:

parentHeader - the parent attribute the new attribute may be anchored to

attributeHeader - the new attribute being tested

Returns:

true if they should be anchored, false otherwise

resolveExperimentFactorTypes

public static Map<String,String> resolveExperimentFactorTypes(MAGETABInvestigation investigation,
                                              HybridizationNode hybNode)

Resolves all the factor value types (the typing information in square brackets - so "foo" in Factor Value[foo]) in the provided hybridization node and maps them to the actual experiment factor type declared in the IDF.

Parameters:

investigation - the MAGETAB investigation this hybNode is present in

hybNode - the hybNode to resolve factor types for

Returns:

a mapping of the factor name (in square brackets in the SDRF) to factor type (declared in the IDF)

escapeCell

public static String escapeCell(String cell)

Performs all necessary escaping for cell values using the following rules: - all TAB characters replaced with spaces - cells containing CR and/or LF characters are quoted using double-quotes - existing quoting is preserved - all double-quote characters inside cell will be escaped by pre-pending backslash - existing escaping is preserved

Parameters:

cell - input cell text value

Returns:

escaped cell text value