# Sparse matrices in python tensorflow text classification

I've been trying to implement a text classification routine using the tensorflow package in python. I already had a successful perceptron version working in the scikit-learn enviroment but scikit-learn does not have multilayer neural networks (except for some mythical version 0.18 that I can't seem to find/install anywhere).

I thought it was best to try something simpler in tensorflow first, to learn how the package works and what it can and cannot do, so I went with nearest neighbors. So far so good, except I just can't find a way to feed a sparse version of the vocabulary matrix (bag-of-words vectorizations of the texts) to a placeholder in tensorflow (in scikit-learn this is no problem at all). Converting the vocabulary matrix to a dense matrix solves the problem but severely slows down the algorithm and clogs up RAM.

Is there any way around this? From what I found on the web it seems tensorflow has very limited support for sparse objects (only certain operations will accept them as input), but I hope I'm wrong.

P.S. Yes, I read this thread and it did not solve my problem. And yes, I know I could stick to the perceptron of the scikit-learn or choose another package, but that's a decision I'll make based on the answers I get here.

## Answers

With TensorFlow 1.0.1, I can do this:

a = sparse.csr_matrix([[0, 1, 2], [5, 0, 0], [0, 0, 5], [10, 1, 0], [0, 0, 4]]) # w = np.arange(6, dtype=np.float32).reshape([3, 2]) a = a.tocoo() a_ = tf.sparse_placeholder('float32') w_ = tf.Variable(tf.random_normal([3, 2], stddev=1.0)) a_sum = tf.sparse_reduce_sum(a_, 1) a_mul = tf.sparse_tensor_dense_matmul(a_, w_) # Initializing the variables init = tf.global_variables_initializer() with tf.Session() as sess: sess.run(init) indices = np.array(zip(a.row, a.col), dtype=np.int32) values = np.array(a.data, dtype=np.float32) shape = np.array(a.shape, dtype=np.int32) print sess.run(a_mul, feed_dict={a_: tf.SparseTensorValue(indices, values, shape)}) w = sess.run(w_) print np.dot(a.todense(), w)

You can find the code from API page: sparse placeholder. After the first layer, other layers of neural network will be dense matrix.