vectorizing a script with cellfun

I'm aiming to import data from various folder and text files into matlab.

clear all
main_folder = 'E:\data';
    %Directory of data
TopFolder = dir(main_folder);
    %exclude the first two cells as they are just pointers. 
TopFolder = TopFolder(3:end);
TopFolder = struct2cell(TopFolder);
Name1 = TopFolder(1,:);
    %obtain the name of each folder
dirListing = cellfun(@(x)dir(fullfile(main_folder,x,'*.txt')),Name1,'un',0);
Variables = cellfun(@(x)struct2cell(x),dirListing,'un',0);
FilesToRead = cellfun(@(x)x(1,:),Variables,'un',0);
    %obtain the name of each text file in each folder

This provides the name for each text file in each folder within 'main_folder'. I am now trying to load the data without using a for loop (I realise that for loops are sometimes faster in doing this but I'm aiming for a compact script).

The method I would use with a for loop would be:

for k = 1:length(FilesToRead);
    filename{k} = cellfun(@(x)fullfile(main_folder,Name{k},x),FilesToRead{k},'un',0);
    fid{k} = cellfun(@(x)fopen(x),filename{k},'un',0);
    C{k} = cellfun(@(x)textscan(x,'%f'),fid{k},'un',0);
end

Is there a method which would involve not using loops at all? something like cellfun within cellfun maybe?

Answers


folder = 'E:\data';
files = dir(fullfile(folder, '*.txt'));
full_names = strcat(folder, filesep, {files.name});
fids = cellfun(@(x) fopen(x, 'r'), full_names);
c = arrayfun(@(x) textscan(x, '%f'), fids);  % load data here
res = arrayfun(@(x) fclose(x), fids);
assert(all(res == 0), 'error in closing files');

but if the data is in csv format it can be even easier:

folder = 'E:\data';
files = dir(fullfile(folder, '*.txt'));
full_names = strcat(folder, filesep, {files.name});
c = cellfun(@(x) csvread(x), full_names,  'UniformOutput', false);

now all the data is stored in c


Yes. This it going to be pretty scary since C depends on fid depends on filename. The basic idea will be:

deal(feval(@(filenames_fids){filenames_fids{1}, filenames_fids{2}, ...
  <compute C>}, feval(@(filenames){filenames, <compute fid>}, ...
  <compute filenames>)));

Let's start with computing the filenames:

arrayfun(@(x)cellfun(@(x)fullfile(main_folder,Name{k},x),FilesToRead{k},...
  'un',0), 1:length(FilesToRead), 'uniformoutput', 0);

this will give us a K-by-1 cell array of filenames. Now we can use that to compute fids:

{filenames, arrayfun(@(k)cellfun(@(x)fopen(x),filenames{k},'un',0), ...
  1:length(FilesToRead), 'uniformoutput', 0)};

We stick fids together with filenames in a K-by-2 cell array, ready to pass on to compute our final outputs:

{filenames_fids{1}, filenames_fids{2}, ...
  arrayfun(@(k)cellfun(@(x)textscan(x,'%f'), ...
  filenames_fid{2}{k},'un',0), 1:length(FilesToRead), 'uniformoutput', 0)}

Then we're putting that final cell array into deal, so that the results end up in three different variables.

[filenames fid C] = deal(feval(@(filenames_fids){filenames_fids{1}, ...
  filenames_fids{2}, arrayfun(@(k)cellfun(@(x)textscan(x,'%f'), ...
  filenames_fid{2}{k},'un',0), 1:length(FilesToRead), 'uniformoutput', 0)}, ...
  feval(@(filenames){filenames, arrayfun(@(k)cellfun(@(x)fopen(x), ...
  filenames{k},'un',0), 1:length(FilesToRead), 'uniformoutput', 0)}, ...
  arrayfun(@(x)cellfun(@(x)fullfile(main_folder,Name{k},x),FilesToRead{k}, ...
  'un',0), 1:length(FilesToRead), 'uniformoutput', 0))));

Errm... There's probably a nicer way to do this if you don't mind about keeping filenames and fid. Maybe using cellfun instead of arrayfun could also make it more concise, but I'm not very good with cellfuns, so this is what I came up with. I think the for loop version is more compact anyway! (also, I haven't actually tested this. It will probably need some debugging).


Need Your Help

Autofilter Excel with VBA

excel-vba access-vba autofilter vba excel

I would like to open Excel from Access and apply filters to a sheet.

@MustOverride annotation?

java interface annotations abstract-class override

In .NET, one can specify a "mustoverride" attribute to a method in a particular superclass to ensure that subclasses override that particular method.