cannot read data file when include staticEM() in a loop

The engine.
Post Reply
Bo Hu
Posts: 18
Joined: Tue Aug 02, 2016 4:13 am

cannot read data file when include staticEM() in a loop

Post by Bo Hu »

Hello,

I encounter some issues when I tried to apply my "parameter learning code" to a number of simulated data samples using a loop. Here is my code.

My simulated data was saved in a .csv file, which include 3000 x 90 binary values and are for 3 data sample. I read it into c++ and then did a loop to make sure each time I will select the 3000 x 30 data file I need and then do the parameter learning.

My code could read in and devide my data correctly. However once I included staticEM() into the main() funciton, things did not work.It shows "Cannot read data file...exiting". The data file (LIN_4.txt) I tried to saved for parameter learning became like below. I do not where the first 0 line came from.

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x20 x21 x22 x23 x24 x25 x26 x27 x28 x29 x30
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 0 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 1 0 1 1 1 1 1 1 0 1 1
1 0 1 0 1 1 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 0
1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

After taking the staticEM() out, everything looks normal and the data file is just like it's supposed to be...

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x20 x21 x22 x23 x24 x25 x26 x27 x28 x29 x30
1 1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1
1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 1
0 1 1 0 1 0 1 1 0 1 0 0 0 0 1 1 1 1 0 1 1 0 1 0 1 1 1 1 1 1

Any helps or suggestions will be appreciated.

Sincerely,
Bo

Code: Select all

#include "stdafx.h"
#define _SECURE_SCL 0
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include <iterator>
#include "smile.h"
#include "smilearn.h"


using namespace std;

	void CreateNetwork(void) {
		DSL_network LIN_4T;
		int x1 = LIN_4T.AddNode(DSL_CPT, "x1");
		int x2 = LIN_4T.AddNode(DSL_CPT, "x2");
		....		
		int x30 = LIN_4T.AddNode(DSL_CPT, "x30");

		int A = LIN_4T.AddNode(DSL_CPT, "A");
		int B = LIN_4T.AddNode(DSL_CPT, "B");
		int C = LIN_4T.AddNode(DSL_CPT, "C");
		int D = LIN_4T.AddNode(DSL_CPT, "D");

		LIN_4T.AddArc(A, x1);
		LIN_4T.AddArc(B, x2);
		.....		
		LIN_4T.AddArc(D, x30);

		LIN_4T.AddArc(B, A);
		LIN_4T.AddArc(C, B);
		LIN_4T.AddArc(D, C);

		LIN_4T.WriteFile("LIN_4T.xdsl");
	}


	void staticEM() {
		
		DSL_dataset LT;
		if (LT.ReadFile("C:\\Users\\Bo\\Desktop\\C++\\LIN_4.txt") != DSL_OKAY) {
			cout << "Cannot read data file...exiting" << endl;
			exit(1);
		}
		DSL_network LIN_4T;
		if (LIN_4T.ReadFile("LIN_4T.xdsl", DSL_XDSL_FORMAT) != DSL_OKAY) {
			cout << "Cannot read network...exiting." << endl;
			exit(1);
		}
		vector<DSL_datasetMatch> matches;
		string err;
		if (LT.MatchNetwork(LIN_4T, matches, err) != DSL_OKAY) {
			cout << "Cannot match network...exiting." << endl;
			exit(1);
		}

		double loglik;
		DSL_em em;
		if (em.Learn(LT, LIN_4T, matches, &loglik) != DSL_OKAY) {
			cout << "Cannot learn parameters...exiting." << endl;
			exit(1);
		}

		LIN_4T.UpdateBeliefs();

                //extracting results and saving into cp_table and then into output.txt
                //this part of code wes deleted to save space
                
		std::vector<double> cp_table(arr, arr + 207);
		ofstream output;
		output.open("C:\\Users\\Bo\\Desktop\\C++\\output.txt", std::ofstream::out|std::ofstream::app);
		output << "\n";
		for (size_t i = 0; i < cp_table.size(); i++) {
			output << cp_table[i];
		output.close();
		}
	}


	int main(int argc, char* const argv[]) {

		CreateNetwork();

		static int data[3000][90];
		std::ifstream file("C:\\Users\\Bo\\Desktop\\Data\\datal1.csv");
		ofstream test("C:\\Users\\Bo\\Desktop\\C++\\test.txt");
		for (int row = 0; row < 3000; row++) {
			std::string line;
			std::getline(file, line);
			if (!file.good())
				break;

			std::stringstream iss(line);

			for (int col = 0; col < 90; col++) {
				std::string val;
				std::getline(iss, val, ',');
				if (!iss)
					break;
				std::stringstream convertor(val);
				convertor >> data[row][col];
			}
		}

		static int dat[3000][30];
		std::vector<std::string> name = {"x1","x2","x3","x4","x5","x6","x7","x8","x9","x10",
			"x11","x12","x13","x14","x15","x16","x17","x18","x19","x20",
			"x21","x22","x23","x24","x25","x26","x27","x28","x29","x30"};

		for (int k = 0; k < 3; k++) {
			ofstream LIN_4("C:\\Users\\Bo\\Desktop\\C++\\LIN_4.txt");
			std::ostream_iterator<std::string> output_iterator(LIN_4, " ");
			std::copy(name.begin(), name.end(), output_iterator);
			LIN_4 << "\n";
			for (int i = 0; i < 3000; i++) {
				for (int j = (k - 1) * 30; j < (k - 1) * 30 + 30; j++) {
					dat[i][j] = data[i][j];
					LIN_4 << dat[i][j] << " ";
				}
				LIN_4 << "\n";
			}
			LIN_4.close();
			staticEM();
		}
	}


shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Re: cannot read data file when include staticEM() in a loop

Post by shooltz[BayesFusion] »

Try adding this statement at the beginning of your main() function.

Code: Select all

ErrorH.RedirectToFile(stdout);
You may get some error messages on the screen after enabling this redirection.

Also, note that DSL_dataset::ReadFile has an overload with 3 arguments, the last one is the pointer to string which gets the parse error message. Change the top of staticEM to this:

Code: Select all

      DSL_dataset LT;
      std::string errMsg;
      int parseCode = LT.ReadFile("C:\\Users\\Bo\\Desktop\\C++\\LIN_4.txt", NULL, &errMsg);
      if (parseCode != DSL_OKAY) {
         cout << "Cannot read data file...exiting" << endl;
         cout << "Error code: " << parseCode << endl;
         cout << "Error message: " << errMsg << endl;
         exit(1);
      }
Bo Hu
Posts: 18
Joined: Tue Aug 02, 2016 4:13 am

Re: cannot read data file when include staticEM() in a loop

Post by Bo Hu »

I got the error message: Trailing separator found in the header line.

Does this violate the format of DSL dataset?
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Re: cannot read data file when include staticEM() in a loop

Post by shooltz[BayesFusion] »

You probably have a trailing space in the first line of your data file.
Does this violate the format of DSL dataset?
Yes.
Post Reply