vec_summaries_reduced and vec_texts_reduced returning blank list []

when i am printing the imported summaries from the pickled file, its alright. It's fully printed in its encoded form. Vec_texts also prints fine. But vec_summaries_reduced and vec_texts_reduced returns blank list.
therefore the train_len is printed as 0.
also Percentage of the dataset with text length less that window size:
99.848
isn't that bad since 99% is less than window size so it is getting reduced?. when d=1 window size 3, then it is 44%, yet no data outputted from vec_summaries reduced ...please tell me what to do?
p.s-> don't know if it has any significance but I had not properly implemented the clean function from your code(it gave some errors). So I only just lowercased everything, nothing else.
update: when i set maxlen_summary as 80 instead of 7, trainlen = 14400. But is setting it as 80 wrong?

Which dataset are you using? Are you using my code?

But vec_summaries_reduced and vec_texts_reduced returns blank list.

Show me the code

isn't that bad since 99% is less than window size so it is getting reduced?.

Yes, that's bad.
It may be possible that you have accidentally switched summaries and text contents. Most summaries are probably less than window size unless d=1 or 2. But window size should be relevant only for vec_texts. So you may check if by chance the content of your vec_text is actually the summaries. But, I don't know. That's just one possibility.

update: when i set maxlen_summary as 80 instead of 7, trainlen = 14400. But is setting it as 80 wrong?

If I remember correctly, I used the maxlen_summary as a limiter to summary length.
So only if summary length < maxlen_summary, I took the summary into consideration, and ignored the summary otherwise.
So again if your summary_vec by mistake actually contains what should be vec_texts then almost no data will satisfy that condition since most (probably all) texts are greater than length 7. Which may be why your vec_summaries_reduced were blank. Since no data satisfied that condition.

Yes I messed up text and summary. Thank you.
Also what classifier did u use if any?

I am not sure what you mean as a 'classifier' here.
In a sense, the whole network is a classifier, and you can think of all the words in vocab_limited as the classes. The immediate decoder output is a probability distribution across all the classes (word).
Then using greedy search, the most highly scored class/word is selected at each timestep in the output.

I meant classifier after reading about handwriting recognition. In that case we have 10 classes, 0....9.
don't you think vocab_limited would be better if it contained most appearing data(based on count).

Usually the network\model itself is the 'classifier' if it's used in a classification problem. For example.
If you use a Neural Network for handwriting recognition, then in that context the neural network is the classifier.
In this case, I am not sure that 'summarization' even is a classification problem - so calling the model used here as a 'classifier' would be inappropriate.
A typical classification constitutes determining the 'type' or 'class' or something of the input.
For example. 0,1,2.......9,A....Z can be thought of as classes. And handwriting recognition can then be thought of choosing in which class a particular handwritten alphanumeric character belongs to.
Sentiment analysis is, for example, a text classification problem. A sentiment analysis program may classify words in a text as 'angry', 'happy', 'sad', 'neutral' or something like that.
Usually classifiction is about predicting something about the input.
In summarization, we aren't exactly telling the characteristics of the input in predicting, but simply creating a summarized version of it.
So, it's probably not a classification problem in the first place. However, you can draw some parallels to the multiclassification process of neural networks if you consider each word in vocabulary as classes (as detailed in my previous post). But overall the words in vocabulary doesn't really have the properties of a 'class', and the whole thing doesn't really seem to be a classification problem.

So each time in the decoder what happens? is it like we are taking probability distribution of the current word with respect to all words in vocab_limit?

probability distribution over all words in vocab_limit means:
Each word in vocab_limit will be assigned a score (the score denotes the probability for the word to be the next word) in such a way that summation of all scores of all the words amounts to 1.
So if there are three words in vocab_limit: Hungry, food and dog.
A probability distribution may look like this:
Hungry: 0.8
food: 0.1
dog: 0.1
In that case, Hungry will be considered as most probable and chosen to be the predicted next word.

But we had already used concatenation in encoding phase to get context between the words in the window?

Window should have no relevance in encoding phase.
I am not which part are you talking about regarding concatenation.
If it's this:

encoded_hidden = tf.concat([hidden_forward,hidden_backward],1)

Then it is simply a method of combing the output of forward and backward encoder.
I used bi-directional encoder.
The forward and backward encoder are part of the bi-directional encoder.
Forward encoder encodes each word in context of the previous words. It starts from the first word and ends at the last word.
Backward encoder encodes each word in context of the later words. It starts from the last word and ends at the first word.
The final result of the bi-directional encoder is the combination of the output of forward and backward encoder.
Here concatenation is used for combining the results.
The result of the concatenation is the final encoder output.

Sorry, still not able to understand as to what information from encoder is used in decoder?

The final encoder output.

#LOCAL ATTENTION
    G,pt = align(encoded_hidden,decoded_hidden,Wp,Vp,Wa,tf_seq_len)
    local_encoded_hidden = encoded_hidden[pt-D:pt+D+1]
    weighted_encoded_hidden = tf.multiply(local_encoded_hidden,G)
    context_vector = tf.reduce_sum(weighted_encoded_hidden,0)
    context_vector = tf.reshape(context_vector,[1,2*hidden_size])

    attended_hidden = tf.tanh(tf.matmul(tf.concat([context_vector,decoded_hidden],1),Wc))

encoder_hidden is the final encoder output.
The window starts from pt-D and ends at pt+D.
The content of the window can be accessed with encoded_hidden[pt-D:pt+D+1].
The window is simply some consecutive words from the encoder output.
local_encoded_hidden contains the window content.
The attended_hidden is calculated indirectly from local_encoded_hidden.
Note: decoded_hidden provides the context of previous decoded words. So the context of previous decoded words (ecoded_hidden) and the context of the final encoder output (encoded_hidden) were used to calculate G, and pt. G, and pt were in turn used to predict the next word after going through some equations.
The next decoder output word is predicted using:

y = tf.matmul(attended_hidden,Ws)

After that the decoder updates the decoder context using RNN (More specifically LSTM).
If a new word is predicted the context of previous decoder words is to be updated for predicting the next word.

decoded_hidden_next,cell_d = decoder(y,decoded_hidden,cell_d,
wf_d,uf_d,bf_d,
wi_d,ui_d,bf_d,
wo_d,uo_d,bf_d,
wc_d,uc_d,bc_d,
RRA)

Yes, I created test data.

train_len = int((.7)*len(vec_summaries_reduced))
train_texts = vec_texts_reduced[0:train_len]
train_summaries = vec_summaries_reduced[0:train_len]
val_len = int((.15)*len(vec_summaries_reduced))
val_texts = vec_texts_reduced[train_len:train_len+val_len]
val_summaries = vec_summaries_reduced[train_len:train_len+val_len]
test_texts = vec_texts_reduced[train_len+val_len:len(vec_summaries_reduced)]
test_summaries = vec_summaries_reduced[train_len+val_len:len(vec_summaries_reduced)]

70% (.7) is training data.
15% (.15) is validation data.
Rest (15%) is test data.

is the value of D found randomnly? d=10, are we taking it as word size (d-1 , d+1)

D is a hyperparameter. D determines the size of the window. It's not completely random. For example, if you randomly put D = 100, window size (2D+1) will be 201, yet most texts aren't that long. So there will be almost no training data left which is greater than the window size. So you can't just randomly put anything. You have to use your discernment. But, there isn't some specific algorithm or rule of thumb to determine the value of D either.
The research paper which introduced local attention (https://arxiv.org/pdf/1508.04025.pdf) used D=10 in some experiments. So, I used D=10 too.

also based on loss value , are you adjusting weights to reduce the loss next time?

Yes. But it is mostly done using Tensorflow functions. So you can't actually see what's happening inside.
These are the codes that specify how the cost is calculated and which optimizer is used during backpropagation.

#OPTIMIZER
cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=output, labels=tf_summary))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

During training, this code performs backpropagation to update weights, calculates loss, and make prediction.

  # Run optimization operation (backpropagation)
        _,loss,pred = sess.run([optimizer,cost,prediction],feed_dict={tf_text: train_texts[i],
                                                tf_seq_len: len(train_texts[i]),
                                                tf_summary: train_out,
                                                tf_output_len: len(train_out)})

"sess.run([optimizer,....]" this part is mainly responsible for backpropagation. The program already knows HOW to adjust the loss, as its defined previously to use Adam Ompitizer. Other processes like differentiation and everything is automated in tensorflow. So all you need to do is sess.run(optimizer) to run backpropagation.

so if d=10, then if we are not filtering non alphabetic characters , they might also be ending up in the window. normal text length is 30 words. so how many layers for that? how would the decoder consider d=10, ie 20 words at a time? we are not inputing it as bag of words rit, but as sequence one word at a time.

again a bit confused, isnt attention process with all the window manipulation taking place during decoding stage...where the tf.concantenate of the forwad ad backward encoders are considered during this time, and window is defined and he context is taken from tht

isnt attention process with all the window manipulation taking place during decoding stage..
More or less.

where the tf.concantenate of the forwad ad backward encoders are considered during this time, and window is defined and he context is taken from tht,

Yes. Window size is predefined (the value of D). Window position is determined at every time step from some formula depending on decoder context (what has been predicted so far) and the encoded context ( the tf.concatenated version of forward and backward encoders). After that attention mechanism is used over the contents of the window (a portion of the encoded text) to create the context vector (which contains the overall attended encoder context from the window) in a single vector. The context vector is then used with the decoder context to make a prediction for the next word. The predicted word is then used to update the decoder context for predicting the next word in the next timestep.

the encoding phase doesnt do much other than going through from sos to eos forward and backward and embedding them rit?

More or less.
Note, I only used the SOS representation as the starting word for the decoder output. I don't think I used SOS symbol anywhere else. But, otherwise, you are right.

i see that you have implemented eos to vec_summary but not for vec_text ....how would you know the end of the text has reached

There can be multiple ways.
You can use len() to find the length of the text. Let's say that the length of text is put into a variable seq_len. The vec_text can be conceived as an array of word vectors.
Then you can simply run a loop from i = 0 to i = seq_len-1.
Psuedocode:
for(i = 0, i<seq_len-1,i++)
{
encode(vec_text[i])
}
eos isn't really used to know the end of summary here. It's main purpose is to teach the program to predict the end for predicted summary. From the eos in the example summary data the model learns where the end of sentence is likely to be while predicting.

def forward_encoder(inp,hidden,cell,                     wf,uf,bf,                     wi,ui,bi,                     wo,uo,bo,                     wc,uc,bc,                     Wattention,seq_len,inp_dim):
the input is a tensorflow placeholder....how are u feeding the placeholder with the data?
also what are does the encoder parameter wf,uf,bf etc .. actually mean? u are creating tf.variable, but what does these paameters actually mean? Also where should I refer to learn more about what you are doing in the forward and backard encoder functions.
thank you

The placeholders are fed during training, validation, and testing.

       # Run optimization operation (backpropagation)
        _,loss,pred = sess.run([optimizer,cost,prediction],feed_dict={tf_text: train_texts[i],
                                                tf_seq_len: len(train_texts[i]),
                                                tf_summary: train_out,
                                                tf_output_len: len(train_out)})

wf, uf, bf are weights and biases for the network.
tf.variables are usually treated as weights and biases for the network. Their values are updated during backpropagation at each training step. Tf.variables are usually treated as the actual trainable parameters.
You may like to take a peek at the introduction to Tensorflow to understand how it works. Some of the things in Tensorflow are fundamentally different from other languages like C, Java, or even base Python itself.
https://github.com/aymericdamien/TensorFlow-Examples

Also where should I refer to learn more about what you are doing in the forward and backard encoder functions.

I used a LSTM with RRA for the encoders. You can ignore RRA.
I implemented my own personal LSTM without using libraries.
To understand LSTM, you can google about it. I can help if you can't understand.
Before LSTM, I will suggest learning about a vanilla RNN, if you don't know already.
You can check this site for RNN:
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
You can check these for LSTM:
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://arxiv.org/pdf/1412.3555.pdf
^check the paper for LSTM and GRU (a slightly simpler version of LSTM in a way). You can find similarities between my code and the mathematical equations as published in it.

thank you. During training in each iteration the loss has to be reduced rit? here during training loss is 10 then 3 then 1 then back to 12 .... so does iterating throughout help much?

Training loss can fluctuate, especially in the early stages. It fluctuates even more here because of no batching. It's not unnatural.
The program may optimize the weights learning from data #1. But in the next training step, the optimization learned from data # 1 may result in increased loss in data #2. That's why we train the model again and again through the multiple data set so that it can ideally finally learn some parameter values that is good for most data set.
If you do batching, each training step will calculate the average loss of all the data in the batch. The average loss probably won't fluctuate as much.
Yes, it needs to be iterated again and again until at least some level of convergence is reach (when the loss only changes negligibly if at all for some training steps). Also, ideally we need to use validation periodically to check for overfitting and such. ( I didn't use validation here because, but it should be used)

Wattention_f = tf.Variable(tf.zeros([K,1]),dtype=tf.float32)
then
Wattention = tf.nn.softmax(Wattention,0)
softmax is ocnverting it into probability distribution rit? but wattention contains tf.zeros[5,1] what does the softmax of that signify.
how many hiddden layers? 500? on what basis is that selected , each state denotes a word?
also
` hidden_forward = tf.TensorArray(size=seq_len,dtype=tf.float32)
hidden_residuals = tf.TensorArray(size=K,dynamic_size=True,dtype=tf.float32,clear_after_read=False)
hidden_residuals = hidden_residuals.unstack(tf.zeros([K,hidden_size],dtype=tf.float32))`

hidden residuals contain the previous 5 hidden states, but how does it know which all states to look at?

You are right that softmax is converting it into a probability distribution.
Wattention_f may be all zeros initially (i.e softmax won't make a difference), but it is a trainable parameter. So its values will be updated in later training change, and then softmax will be necessary.
Also, Wattention part in encoders is a part of the RRA mechanism, which you may ignore.
hidden_forward is like a list which will contain all the hidden states of the forward encoder. The hidden states are the encodings. So you can say that the hidden_forward will store all the encoded words resulted from the forward encoder. Hidden_backward is the same but for backward encoder. They are dynamic lists. First they are empty, but they are filled with hidden states as they are generated by the encoder. seq_len (no. of words in the sequence fed into the encoder) is the max size, since the no. of encoded words will be same as the seq_len.
hidden_size = 500 ....hidden_size denotes the dimension size of each hidden states that are being used throughout the encoders. Decoders will be using twice the size of encoder hidden size. Decoder hidden size will be 100. Another way to think of hidden_size is that it denotes the number of neurons in the hidden layer.  Since the encoder hidden state will be treated as the encoded versions of the input words, hidden_size denotes the dimensions of the encoded word vector output from each encoders.
The final output of the encoder after concatenating the forward encoder output and backward encoder output will be then of course of size 500+500 = 1000. I initialized the decoder hidden state with the first encoder hidden state (of size 1000). That is why decoder hidden size is also kept at 1000.
There is no strict science behind choosing 500 for hidden size. Hidden size, These are hyperparameters that are initialized somewhat based on intuitions and then some trial and error are used. At best if you have enough processing power you can use some code to randomly generate a lot of hyperparameters and test the first couple of results and then choose the set of hyperparameters that gives the best result.
You may use 100, 200, 300 or such for hidden_size. Using greater size means there will be more dimensions to store more encoded information, which may result in better result. But that can also mean slower processing, or storing useless information which may result in bad result. So it's tricky. The best we can do as a beginner is refer to how the experts initialize the hyperparameters for similar programs. Research papers about different models often come with details about the values that were used for hyperparameters and so on.
Hidden_residuals is again a part of RRA which you may ignore.
The logic behind the codes for hidden_residual and RRA is a bit complicated. RRA itself is not complicated but implementing it in Tensorflow was a bit complicated for me, due to some limitations of Tensorflow. So explaining it will take a bit of time. I can try to explain it later if you want to use RRA, but for now I can give a brief description.
Hidden_resiudals is roughly used similar to the hidden_forward or hidden_backward tensorarrays which are lists that are updated with the hidden states\encoded words as they are generated. In the end they will contain roughly all encoded words.
RRA will be using last five hidden states. The last five hidden states (placed in the last 5 filled indices) of hidden_residual will be used for that.
K denotes how many previous hidden states are to be used for RRA. Here, K=5.
J is initialized as 5. So at first step by iterating from j-k to j-1 (0 to 4) numbered indices from hidden_resuiduals you can find the first 5 hidden states for RRA. At each encoding step J is increased by one while a new hidden state is also added to the list of hidden_residuals. So at the next step j will be 6, and K will be 5, so iterating from j-k to j-1 indices in hidden_residuals is like iterating from index 1 to index 5 where index 5 will contain the newly added hidden state. In the next step J will be 7 and K will be 5. so from j-k to j-1 or j-k:j will be from 2 to 6...and so on. In this way only the last 5 or last K hidden states which will be located in the last 5 filled indices of hidden_residuals will be used for RRA.

` def body(i,j,hidden,cell,hidden_backward,hidden_residuals):
    x = tf.reshape(inp[i],[1,inp_dim])

    hidden_residuals_stack = hidden_residuals.stack()

    RRA = tf.reduce_sum(tf.multiply(hidden_residuals_stack[j-K:j],Wattention),0)
    RRA = tf.reshape(RRA,[1,hidden_size])

    # LSTM with RRA
    fg = tf.sigmoid( tf.matmul(x,wf) + tf.matmul(hidden,uf) + bf)
    ig = tf.sigmoid( tf.matmul(x,wi) + tf.matmul(hidden,ui) + bi)
    og = tf.sigmoid( tf.matmul(x,wo) + tf.matmul(hidden,uo) + bo)
    cell = tf.multiply(fg,cell) + tf.multiply(ig,tf.sigmoid( tf.matmul(x,wc) + tf.matmul(hidden,uc) + bc))
    hidden = tf.multiply(og,tf.tanh(cell+RRA))

    hidden_residuals = tf.cond(tf.equal(j,seq_len-1+K),
                               lambda: hidden_residuals,
                               lambda: hidden_residuals.write(j,tf.reshape(hidden,[hidden_size])))

    hidden_backward = hidden_backward.write(i,tf.reshape(hidden,[hidden_size]))

    return i-1,j+1,hidden,cell,hidden_backward,hidden_residuals

_,_,_,_,hidden_backward,hidden_residuals = tf.while_loop(cond,body,[i,j,hidden,cell,hidden_backward,hidden_residuals])`

This code is the most troublesome for me. Fromt he paper which you gave me i understood that fg, ig,og are the forget gates , input gate and the output gate?
When is the body function executed? Is it when forward_encoder function is called?  x contains the inp[i], ie for 0....end? In the model function wf,uf etc are just initlialised, but how are their values set?
"cell" in this code follows the cell formula from the paper which is partially forgetiing the existing content and adding new memory.
 hidden = tf.multiply(og,tf.tanh(cell+RRA))
hidden over here, is it the hidden state corresponding to each input?  what exactly can the tanh activation function do to the values, it makes it non linear...but in the context of text, what does tanh do?
Thank you

From the paper which you gave me i understood that fg, ig,og are the forget gates , input gate and the output gate

Yes.

When is the body function executed? Is it when forward_encoder function is called?

No. This line within the forward_encoder will execute cond and body.

,,,,hidden_backward,hidden_residuals = tf.while_loop(cond,body,[i,j,hidden,cell,hidden_backward,hidden_residuals])`

This is just the 'Tensorflow's' way of implementing dynamic loops (as in loops that depends on some dynamic value i.e values that are specified during runtime)
Check: https://www.tensorflow.org/api_docs/python/tf/while_loop
Also, check this: https://stackoverflow.com/questions/37441140/how-to-use-tf-while-loop-in-tensorflow

x contains the inp[i], ie for 0....end?

Yes. X will contain the input word vector value. In the first loop it will start with the 1st word (i=0), in the next loop i will be increased (i++) and so it will contain the 2nd word vector and so on till the end.
The code you quoted is from the backward encoder where i starts from the end and continues to 0, i's value being decremented every iteration.

In the model function wf,uf etc are just initlialised, but how are their values set

First codes like this:

wf_f = tf.Variable(tf.truncated_normal(shape=[word_vec_dim,hidden_size],stddev=0.01))
uf_f = tf.Variable(np.eye(hidden_size),dtype=tf.float32)
bf_f = tf.Variable(tf.zeros([1,hidden_size]),dtype=tf.float32)

This doesn't actually initializes the value, instead it specifices how to be initialized. Or you can say it specifies the structure of the variable too.
For example tf.truncated_normal will initialize the values using random normal distrubition of stddev=0.01 while truncating some values. np.eye will initialize a matrix identity of the given shape. And so on.
This code in the training section:

init = tf.global_variables_initializer()
with tf.Session() as sess: # Start Tensorflow Session
sess.run(init) #initialize all variables

Will actually initialize i.e set the initial values.
The values are then adjusted by Tensorflow during backpropagation.

cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=output, labels=tf_summary))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

The optimizer defines how to adjust the values. The minimize(cost) part will differentiate the cost and backpropagate to find the gradients of all trainable parameters, and then adjust those parameters using the specific optimization method trying to minimize the cost. All done with Tensorflow.
Note, the above code is more like a definition of how optimization is to be done. Executing it does not initiate the optimization process or backpropagation.
This code starts the backpropagation. Not how 'optimizer' is present within the sess.run.

        # Run optimization operation (backpropagation)
        _,loss,pred = sess.run([optimizer,cost,prediction],feed_dict={tf_text: train_texts[i],
                                                tf_seq_len: len(train_texts[i]),
                                                tf_summary: train_out,
                                                tf_output_len: len(train_out)})

"cell" in this code follows the cell formula from the paper

Yes.

hidden over here, is it the hidden state corresponding to each input?
Yes.

it makes it non linear...but in the context of text, what does tanh do?

Well, we are dealing with word vectors not raw texts. A word vector is a vector - which is like an array of numbers. All the previous equations are just linear algebras. So even if we are dealing with texts it's not all too different from dealing with an image (which is represented as a tensor).
So, in the end we are dealing with numbers. Tanh is still used for non-linearity here.

  RRA = tf.reduce_sum(tf.multiply(hidden_residuals_stack[j-K:j],Wattention),0)         RRA = tf.reshape(RRA,[1,hidden_size])
what does this rra do? It finds the k hidden states and what does the weighted sum mean? I thought rra was used for local attention during comparing decoder with encoder states. It had d as window size. But here k hidden states are taken. How does this relate to local attention model specified later?

RRA is Recurrent Residual Attention (https://arxiv.org/abs/1709.03714).
It is different from local attention.
It doesn't have any special relation with local attention.
I recommended earlier that you can avoid RRA and any codes associated with it, completely. I discovered the concept of RRA in the aforementioned paper. The paper isn't very highly cited, and I am overall suspicious of the quality of the method.
Local Attention is in a sense an interlayer attention. It's something that can be said to happening between encoder and decoder. Yes, local attention is being used during decoding, but the local attention takes both encoder states and decoder states in account. That is two separate layers are in communication for local attention. Which is why it can be considered as an interlayer attention.
RRA, in contrast, is more of an intra-layer attention. Here attention is happening among the states of the same layer. In encoder, RRA is used for attending to previous hidden states of the same encoder layer. In the decoder, RRA is again used for attending to previous decoder hidden states of the same layer.
However the concept of inter-layer attention, seems to me, to be more fleshed out in the LSTMN model.
https://arxiv.org/pdf/1601.06733.pdf
So I will recommend against using RRA. If you really want to use some intralayer attention you can try LSTMN.

It finds the k hidden states and calculates the weighted sum but what does it mean

It finds the 'last' K hidden states. The K is a predefined hyperparameter.
The K hidden states are then multiplied by K weights (trainable parameters).
(i.e each of the K hidden states are multiplied with one of the K weights).
The result of the multiplication is a list of 'weighted hidden states'.
The same of these weighted hidden states is the weighted sum.
Example:
list of 1D vectors = [0],[2],[3]
list of weights = 99, 5, 10
Weighted sum of the 1D vectors = 99*[0] + 5*[2] + 10*[3] = 40

hidden_residuals_d = tf.TensorArray(size=K,dynamic_size=True,dtype=tf.float32,clear_after_read=False)     hidden_residuals_d = hidden_residuals_d.unstack(tf.zeros([K,2*hidden_size],dtype=tf.float32))
The rseiduals are stacked layer after another, so as to get the most immediate output? But when its unstacked it returns a list? so what exactly does unstack mean in this context?
would classify this project as deep learning?
All the methods for finding out pt and context vector can be understood from the paper ri?
but what is y used here for
  y = tf.convert_to_tensor(SOS) #inital decoder token <SOS> vector     y = tf.reshape(y,[1,word_vec_dim])
then its used as
` y = tf.matmul(attended_hidden,Ws)
    output = output.write(i,tf.reshape(y,[vocab_len]))

    y = tf.nn.softmax(y)

    y_index = tf.cast(tf.argmax(tf.reshape(y,[vocab_len])),tf.int32)
    y = tf_embd_limit[y_index]
    y = tf.reshape(y,[1,word_vec_dim])`

Thank you

The rseiduals are stacked layer after another, so as to get the most immediate output? But when its unstacked it returns a list? so what exactly does unstack mean in this context?

Hidden residuals is a part of RRA. It is used to store all pervious hidden states, and from this list last K hidden states are obtained. If you don't want to bother with RRA, you can ignore hidden_residuals.
Read about what stack and unstack is from here:
https://www.tensorflow.org/api_docs/python/tf/TensorArray
Remember hidden_residuals is a Tensorarray. It's a kind of dynamic list with some specific rules to read or write data. .stack() simply obtains a Tensor made of the data in Tensorarray. A Tensor can be more conviniently handled for accessing data. I don't remember the exact reason I used stack() to obtain a Tensor. May be I faced some issue when using Tensorarray to access data, or may be I did it because I was more convenient.
Unstack does the opposite. It puts the values from a Tensor into the indices of the Tensorarray.
I used:
hidden_residuals_d = hidden_residuals_d.unstack(tf.zeros([K,2*hidden_size],dtype=tf.float32))
to basically initialize the Tensorarray with 0.
tf.zeros(K,2*hidden_size) is a tensor of K elements, each having a dimensions of the decoder hidden state, and all the values are 0.
At the beginning the Tensorarray hidden residuals will thus contain a list of K elements all having 0 throughout its dimension.
So the first time the program requires the last K hidden states, it will only get 0 valued hidden states since there is no hidden state during the first time. Being zero they won't have any effect on the calculation of RRA.

would you classify this project as part of deep learning?

Yes. Pretty much anything with a multilayered neural net is deep learning. Recurrent Neural Network (RNN) is a type of Neural Network, and LSTM which I used is a type of RNN. Like a typical deep learning project it contains stacked layers of neurons arranged in a certain way. Here you find how decoder layers are stacked after encoder layers with some attention in between.

All the methods for finding out pt and context vector can be understood from the paper ri?

If you mean the context vector and pt from local attention, then yes.
It's here: https://arxiv.org/abs/1508.04025

but what is y used here for
y = tf.convert_to_tensor(SOS) #inital decoder token  vector y = tf.reshape(y,[1,word_vec_dim])
then its used as
` y = tf.matmul(attended_hidden,Ws)

y signifies the decoder output of a decoder time step.
SOS is a word vector that signifies the start of the sentence.
You missed the use of y after initializing y with SOS.
Y was used here before y = tf.matmul(attended_hidden,Ws):

decoded_hidden_next,cell_d = decoder(y,decoded_hidden,cell_d,
wf_d,uf_d,bf_d,
wi_d,ui_d,bf_d,
wo_d,uo_d,bf_d,
wc_d,uc_d,bc_d,
RRA)

Here y which is SOS in the first time step provides the initial context for predicting the next word (which will be the actual first word).
After this, the work of SOS is done.
Y is updated with the newly predicted word-vector.
Take note, the decoder function uses the previous decoder predicted word to predict the next word.
Y provides this previously predicted word. So at each time step Y is updated with the last predicted word.
Since in the beginning there was no predicted word, Y was initialized with SOS which provides the start of sentence context.

also what is the concept behind alignment? How is allignment significant?

I don't understand what you mean by aligment?
Are you talking about this?:

G,pt = align(encoded_hidden,decoded_hidden,Wp,Vp,Wa,tf_seq_len)

Alignment here is basically a part attention. The align function determines the pt i.e the center position of the window when attention will be given, and it also determines G which has the weights of the encoded words within the window - the weights signifies how much attention is to be given.
It's named align because in a sense it is aligning the encoded words with the decoder by determining where to attend and how much to attend to the encoder in order to get the appropriate context for decoding.

for vec in train_texts[i]:                     if vec2word(vec) in string.punctuation or flag==0:                         print(str(vec2word(vec)),end='')                     else:                         print((" "+str(vec2word(vec))),end='')                     flag=1
converting this vector to word, but what does the end and string.punctuation achieve in this context? Also, what is the use of transform_out function?
also
`def cond_pred(i,pred):
return i<tf_output_len
def body_pred(i,pred):
pred = pred.write(i,tf.cast(tf.argmax(output[i]),tf.int32))
return i+1,pred
i,pred = tf.while_loop(cond_pred,body_pred,[i,pred]) `
how is data fed into the pred? what exactly does the argmax of output[i] do? output would be the prediction rit, then what?
thank you

I don't know exactly what string.punctuations is; it's probably a list of punctuations in string format.
I know that if you check " if string in string.punctuation" -it will return true if the string is a punctuation.
So using this condition you can check wether the current word is a punctuation or not.
This part of the code is probably not necessary here - because I filtered out punctuations IIRC. But if you are making a more sophisticated program that takes punctuations into account, then this code can be useful.
Basically this code is for formatting the display of the output.
flag is 0 only for the first word, I believe.
So if vec2word(vec) in string.punctuation or flag==0 will return true only if the word is a punctuation or the word is the first word.
specifying end='' within the print() means, the printed element will end with an empty string '' i.e it will not end with anything.
By default the end = '\n' or ' ', IIRC. That is if you simply code "print("xyz")" the actual printed output will be xyz followed by a newline or blankspace.
So if we are dealing with a first word or punctuation, it is followed by nothing and no space is given before it.
Otherwise a space is put before the word.
print((" "+str(vec2word(vec))),end='')
See how a space is concatenated before the word.
So what is happening?
Let us say we have a set of words\punctuations like this:
'Jack' ',' 'Jill' ',' 'and' 'I' 'are' 'three' 'guys' '.' ( Jack, Jill, and I are three guys.)
The program will start with Jack. At that time flag will be zero. So it will simply print Jack with nothing following it.
Output at this point: "Jack"
Next word will be actually not a word but a punctuation ",". So following the code it will again simply print "," without any space before or after.
Output at this point: "Jack,"
Next word is "Jill" which is neither a punctuation nor the first word, so it will concatenate a blank space before Jack and print it.
Output at this point: "Jack, Jill"
....and so on.
Now, you may understand why I did what I did.

what is the use of transform_out function

It changes the target output (or label) format.

But to use Tensorflow's  sparse_softmax_cross_entropy_with_logits() function for loss calculation,
the target output\labels needed to be in a special format.
The output needed to be integers signifying the position of the vocabulary (which can be conceived as a sort of label no.).
A slightly more detailed explanation is given above the transform_out function in the readme file.
You may have a easier time understanding if you know one-hot-encoding.

how is data fed into the pred?

Initially pred is an empty Tensorarray with a size defined. This tensorarray object - pred is initially fed into the loop.
WIthin the loop, pred is updated with information from which the predicted word can be deduced.
At the end of the loop pred will have information denoting all the predicted word.

argmax of output[i]

Output[i] contains the probability distribution of all the words in vocab_limit for position i of prediction.
In other words, output[i] contains the probability of each word in vocab_limit to be position i.
tf.argmax(output[i]) will return the index with output[i] that contains the maximum probability.
This index also signifies the position of the word in vocab_limit.
(jth value in output[i] will have the probability of the jth word in vocab_limit to be in the ith position)
So if you know the index, you can easily know the word by extracting it from vocabulary.
Basically argmax is used to choose the index of the maximum probability value so that we can finally find the word that is given most probability to be in the ith position.
This index information is stored in pred. Later on, the word can be retrieved from the index to display the predicted word.

So in decoder, using encoder hidden states, previous decoder state and the context are you trying find out the word in vocab_limit that contain the max probability and directly printing that word as predicted summary?
train_out = transform_out(train_summaries[i][0:len(train_summaries[i])-1])
what is the use of train out, is it only to print which is the actual summary and its length? does it have any other use?
cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=output, labels=tf_summary))
here is logits is softmaxed then compared with one hot encoded label....where did one hot encodin of tf_summaries take place(since its just a place holder) ?

So in decoder, using encoder hidden states, previous decoder state and the context are you trying find out the word in vocab_limit that contain the max probability and directly printing that word as predicted summary?

Yes.

train_out = transform_out(train_summaries[i][0:len(train_summaries[i])-1])
what is the use of train out, is it only to print which is the actual summary and its length? does it have any other use?

train_out contains the value returned by transform_out.
As I already said transform_out converts the target_output into a specific format such that it can be used in Tensorflow's sparse_softmax_cross_entropy_with_logits() function for loss calculation.
That's why train_out is fed to the network for loss calculation.

# Run optimization operation (backpropagation)
        _,loss,pred = sess.run([optimizer,cost,prediction],feed_dict={tf_text: train_texts[i],
                                                tf_seq_len: len(train_texts[i]),
                                                tf_summary: train_out,
                                                tf_output_len: len(train_out)})

Check above. tf_summary is fed with the values in train_out.

here logits is softmaxed then compared with one hot encoded label....where did one hot encodin of tf_summaries take place(since its just a place holder) ?

One hot encoding was not used. But something similar was used.
For example, let's say all the words in the vocabulary are possible labels. let's say there are 3 words.
So if the target output is the 2nd word..in one hot encoding it will be 010.
so in index 0 there is 0, in index 1, there is 1, and in index 2 there is 0.
softmax_cross_entropy_with_logits() will be satisfied with this format.
But if you use sparse_softmax_cross_entropy_with_logits(), instead of 010 you have to feed it with 1.
Why 1?  because the 1 of the one-hot format is present in index 1. If the one hot encoding was 001, you had to feed 2. Basically, you are feeding the position of the word in the vocabulary i.e the position where the 1 would have been present if it was one hot encoded.
Target output words are transformed into this format (that contains the index of the word in vocab_limit) within the transform_out function.

Thank you.
Attention finds the maximum probable word or context form the window, then its used along with the decoder state....but how does vocab_limit comes into this? Since we can directly find the most probable word using attention..... earlier u mentioned about creating probability distribution in vocab_limit but we are already finidng the context vector using attention from the training text list. Vocab_limit would be haing way too many words ...like the full vocab... is the decoder using the context along with words in vocab_limit?  can you  please explain that concept? also how does attention window go beyond len-1, restricted to use ot-d,pt+d as long as pt is not len-1, so what would it contain if pt+d goes beyond len-1?

Attention finds the maximum probable word or context form the window, then its used along with the decoder state

No.
Finding maximum probable word from the probability distribution is something that happens at the end - at the time of making final predictions.
Attention doesn't have anything to do with it.
Local Attention here helps to direct attention towards encoder states for predicting the decoder word (probability distribution - from which much later the maximum probable word is chosen)

but how does vocab_limit comes into this

vocab_limit simply contains all the words in the vocabulary. It's actually a limited vocabulary, because vocab_limit only contains word present within the training data.

Since we can directly find the most probable word using attention
I am not sure where you are getting that idea.

I was describing the selection of most probable word while describing the code with pred, and the work of argmax. Those codes happen after attention and decoding are complete. So I am not sure why you are associating attention with this.

earlier u mentioned about creating probability distribution in vocab_limit but we are already finidng the context vector using attention from the final encoded states

The context vector from attention is USED to predict the probability distribution OVER the words in vocab_limit. The probability distribution is the initial decoder output. This is also used for loss calculation.
Later max probable word is chosen from the distribution as the final prediction to display.

Vocab_limit would be haing way too many words ...like the full vocab... is the decoder using the context along with words in vocab_limit?
Yes vocab_limit has too many words. The processing would be heavier if previously decoded words used for context were all probability distributions over vocab_limit. So first the probability distribution is saved in output tensorarray for loss calculation later on. Then the word vector of the max probable word is chosen to serve as the context for previous decoded word. So yeah, this is another place where the max probable word is used, before the final prediction. Check the codes:

  output = output.write(i,tf.reshape(y,[vocab_len]))
    #Save probability distribution as output

    y = tf.nn.softmax(y)

    y_index = tf.cast(tf.argmax(tf.reshape(y,[vocab_len])),tf.int32)
    y = tf_embd_limit[y_index]
    y = tf.reshape(y,[1,word_vec_dim])

also how does attention window go beyond len-1, restricted to use ot-d,pt+d as long as pt is not len-1, so what would it contain if pt+d goes beyond len-1?

The problem is coded in such a way that pt+d would never go beyond len-1.
Check this:

positions = tf.cast(tf_seq_len-1-2*D,dtype=tf.float32)

sigmoid_multiplier = tf.nn.sigmoid(tf.matmul(tf.tanh(tf.matmul(ht,Wp)),Vp))
sigmoid_multiplier = tf.reshape(sigmoid_multiplier,[])

pt_float = positions*sigmoid_multiplier

pt = tf.cast(pt_float,tf.int32)
pt = pt+D #center to window

sigmoid_multiplier's range is from 0 to 1 because sigmoid(x)'s range is 0-1.
So positions*sigmoid_multiplier i,.e float version of pt ranges between 0 to positions.
positions is set up to be of size len-1-2*D
So finally the range of pt is 0 to len-1-2*D
after that there is this code pt=pt+D
That means the range of pt becomes D to len-1-D
So pt-D at minimum = 0
and pt+D at maximum = len-1
So pt-D can never be less than 0, and pt+D can never exceed len-1.

How is The context vector(average) from the encoded words compared with vocab_limit. That is what i am having trouble with. Since vocab_list can have anything...say context vector meaning is "lion".... the vocab_limit can have say "cat","tiger" etc..... how is the context vector used to find max probable word since its not similar to anything in vocab_list. Please bear with me, just not able to understand that concept.

I can't clearly understand your question. It seems to me you may be misunderstanding something critical, but I can't understand what it is from the question. Nevertheless, I will try to answer to the best I can.
I am assuming by context vector you mean this:

    weighted_encoded_hidden = tf.multiply(local_encoded_hidden,G)
    context_vector = tf.reduce_sum(weighted_encoded_hidden,0)
    **context_vector** = tf.reshape(context_vector,[1,2*hidden_size])

This context vector isn't exactly an average. It's a 'weighted summation'. Check back on my description of the weighted summation, if you need.
Context_vector typically won't have any concrete meaning such as 'lion'.
We can't really think in precise human terms of meaning of words since we start encoding.
Even during encoding we are associating context information.
The encoded version of the word 'lion' won't simply mean 'lion', it will be lion+some context information.
Things get too abstract here.
Context vector is going even further than that - it's the weighted summation of these already contextually encoded versions of the input words.
Let's say we have encoded version of 'lion', and 'tortoise'...the attention process may assign .75 attention weight to lion and .25 attention weight to 'tortoise'. Then the context vector will be .75 * encoded vector of 'lion' + .25 * encoded vector of 'tortoise'....so at best you can say that the context vector is mostly lionish and slighlty tortoisish....
The thing is a well trained network will give higher attention weights to important encoded words - and thus the context vector will be closer in meaning to the important words...the idea is to get the context vector to represent the essence or 'gist' of the encoded info.

the vocab_limit can have say "cat","tiger" etc

yes.

how is the context vector used to find max probable word since its not similar to anything in vocab_list.

The context vector is not directly used to find the max probable word.
You can check the codes for what is exactly happening.
A brief overview of what's happening (without maths\codes):
Some linear algebra and non-linearity is used (multiplying with weights and all that) to transform the context vector, and other important information (previous decoder word, previous decoder hidden state etc.) into a data that is actually a vector of same dimensions as the no. of words in the vocabulary.
This data (after softmax) is then treated as the probability distribution over the words in the vocabulary.
That's the beauty of linear algebra operations. You can transform data of any dimension to another data of any dimension.
For example, handwritting recognition. You give in high dimensional data (tensor representation of the images) and it is converted into a single digit representing its class.
Note, we are not directly predicting 'cat' or 'dog'. We are initially predicting a probability distribution over the words. So instead of cat or dog, the program may predict 0.7, 0.3 - this data then can be treated as if there is a 0.7 probability for it to be a cat, and 0.3 probability for it to be dog in the given position.
Before training the prediction will be just random numbers. After training ideally the network will learn to use the important information from context vector,  previous decoder prediction etc. to result in a set of data that can better serve as a probability distribution.
Now, once you have the probability distribution finding the max probable word is a piece of cake.
You can find the index where the probability value is maximum, and then just output the word in the same index within vocabulary. (The data is here treated in such a way that the value in ith position is considered as the probability for the ith word in vocab).

Please bear with me, just not able to understand that concept.

It's fine.

Thank you. Also tarining_iter =5 but iteration goes on till 25000 approx...so what does training_iter do

while step < training_iters:

    total_loss=0
    total_acc=0
    total_val_loss = 0
    total_val_acc = 0

    for i in xrange(0,train_len):

        train_out = transform_out(train_summaries[i][0:len(train_summaries[i])-1])

        if i%display_step==0:
            print("\nIteration: "+str(i))

what does step do?

I used training_iter as an epoch, I guess. My phrasing was poor. That is, 1 training_iter is completed when all training data is used for training for once. Since there are lots of data, actually training iterations will be above 1000s.

Are you aligning the pred output with the actual summary? In translation i think they align target output with actual output....or is the actual summary just used to find out loss? If the probabily distribution of each decoder timestep is found..then why are some predicted summaries having only one word?
also, I didn't properly understand the output[i] part...you are using write(i,tf.cast(y....)) are you putting y to ith location and later accesing it as output[i].....

Context vector is not decoder output, nor is it the sole determining factor for the decoder output. You can at best say that context vector carries the gist of an attended portion of the encoded vectors or something like that. As you can see the code for details.
The decoder output is put into y. The value of y gets updated every timestep. We need something to STORE the predicted values (probability distributions)'decoder outputs to use it later for loss calculation and prediction, Output serves this purpose. Output is a tensorarray where the new value of y ( new decoder output_ of a timestep) is stored.
The deocder may output multiple probability distributions, since the output can have multiple words. At each timestep a probability distribution for a new position in the output sentence is predicted.
outpout[i] simply signifies the probability distribution for predicting the word to put in position i in output sentence.

how can we know ith element in vocab_limit exactly..

use vocab_limit[i]

Are you aligning the pred output with the actual summary? In translation i think they align target output with actual output....or is the actual summary just used to find out loss?

By aligning do you mean comparing between the predicted output and actual summary to find the loss so that the program can optimize the trainable weights in order to make predicted output be closer to the actual output?
I am not sure what you mean by 'actual output', and 'target output'.
There's a predicted output that the model predicts. The prediction is the actual predicted output.
There's a target output  which is what we want the machine to predict. It's can be considered as the true output or actual output as it is what we think to be the correct desirable output. This is what the machine strives towards.
If by actual output you mean target output, what do you mean by aligning target output with actual output? Wouldn't they be the same?
So I don't get what you are asking for.
But, all in all, things here aren't different from translation using encoder-decoder.
In translation we compare the model-predicted probability distributions of the translation with the actual target translation.
In summarizsation, we compare the model-predicted probability distributions of the summarization with the actual target summarization.

If the probabily distribution of each decoder timestep is found..then why are some predicted summaries having only one word?

Those output is the result of one decoder timestep. One probability distribution was determined obnly. So only one max probable output was retrieved. The putput size is predefined by user in this program. But a better program should learn to predict the end of the output by itself. In those case the end is marked by a vector representation of another symbol  (end of sentence). If the program predicts an  everything afterwards should be ignored while displaying or loss calculation.

also, I didn't properly understand the output[i] part...you are using write(i,tf.cast(y....)) are you putting y to ith location and later accesing it as output[i].....

Yes.

Are you aligning the pred output with the actual summary? In language translation i think they align target output with actual output....or is the actual summary just used to find out loss? does the size of actual summary have any consequence on the output length or its characteristic.

sorri repeated 2 questions...it happened due to internet error.

does the size of actual summary have any consequence on the output length or its characteristic.

Normally it shouldn't. But here I used the size of the target summary to predefine the size\length of the predicted output  which is also the no. of decoder timesteps. It was for testing purposes - to make the program simpler.

so LSTM is doing its remembering and forgetting part by transforming the values ..ie

cell = tf.multiply(fg,cell) + tf.multiply(ig,tf.sigmoid( tf.matmul(x,wc) + tf.matmul(hidden,uc) + bc))

by multiplying it with fg,ig.... the original values are transfomed to represent something else?

hidden_forward = hidden_forward.write(i,tf.reshape(hidden,[hidden_size]))

here write would write the value of hidden(say [0.2 0.4 0.5....] to i ? so 0th element would represent a particular word which is a 500 dimensional vector?  So if it was Lion, now its transformed to a new entity and placed on 0th index?

the original values are transfomed to represent something else?

Yes. But it doesn;'t necessarily represent something else entirely. It can be hard to say what exactly it represents. It's supposed to represent mixture of various data (input data, previous cell state data, hidden state data etc).

here write would write the value of hidden(say [0.2 0.4 0.5....] to i ?

Yes.

so 0th element would represent a particular word which is a 500 dimensional vector? So if it was Lion, now its transformed to a new entity and placed on 0th index?

Note 'lion' was transformed to a vector (in this case 50 dimensional vector) long before the network starts. IT was done in early preprocessing stage using GloVe.
The encoder LSTM transforms it again into a 500 dimensional vector using all the formulas with ig, og etc. This 500 dimensional vector can be said to be the encoded version of the 50 dimensional word vector.
Usually context information (from previous words) are encoded with the 50 dimensional word during its transformation to the 500 dimensional vector.
So the 0th index will contain the context encoded 500 dimensional vector version of the 50 dimesnional GloVe word vector representation of lion.

RRA = tf.reduce_sum(tf.multiply(hidden_residuals_stack[j-K:j],Wattention_d_normalized),0)
RRA = tf.reshape(RRA,[1,2*hidden_size])

decoded_hidden_next,cell_d = decoder(y,decoded_hidden,cell_d,
                              wf_d,uf_d,bf_d,
                              wi_d,ui_d,bf_d,
                              wo_d,uo_d,bf_d,
                              wc_d,uc_d,bc_d,
                              RRA)
decoded_hidden = decoded_hidden_next

hidden_residuals_d = hidden_residuals_d.write(j,tf.reshape(decoded_hidden,[2*hidden_size]))

this is before entering the while_loop in decoder. We are getting encoded_hidden[0] vector and inital token SOS as y.  Then setting j=5( ie k) , first case rra would be zeros since there is nothing in hidden_residual_stack. But after calling the decoder function now the value of "decoded_hidden" is "decoded_hidden_next" , but why is that value written to j ?  j=5 ..... there wont be anything in residual stack from 0 to j-1, then why is the value given to only j?

but why is that value written to j ? j=5 ..... there wont be anything in residual stack from 0 to j-1, then why is the value given to only j?

At first hidden residuals had nothing just (nd-arrays of 0s).
After calling the decoder function, we have a new decoder hidden state (decoded_hidden_next) which is stored in decoded_hidden (which will be used as one of the inputs to the decoder function in the next loop).
Hidden_residuals is supposed to store the last 5 hidden states. We don't have 5 yet, but we still have 1. So we store this 1 hidden state (decoded_hidden) to hidden_residuals_d.
So all afer calling the decoder function for the first time, all hidden residuals values should be zeros except one (which should contain the new decoded_hidden value)
I stored it at j because I was following a certain logic.
In the beginning, hidden_residuals_d has 5 elements (0 to j-1 where j=5) each being 0 vectors.
To access the last five hidden_residuals I used:

RRA = tf.reduce_sum(tf.multiply(hidden_residuals_stack[j-K:j],Wattention_d_normalized),0)

hidden_residuals_stack[j-K:j] will return the array of values from position j-K to j-1 (yes, j-1 not j).
since initially j=5 and K=5, it will be 0 to 4 (the first 5 0 vector values)
After the first decoder hidden state is determined, it is stored in the 6th position i.e 5th index (starting from 0). So then hidden_residuals_d will contain 6 elements.
The first 5 will be 0 vector values. The sixth one will be the new decoder hidden state (at index 5).
For the next iteration j is increased by 1. So now j will be 6.
Again the last 5 (K) hidden states will be accessed using hidden_residuals_stack[j-K:j]
that is from value at index j-K (which is 6 - 5) to j-1 (which is 5).
So we will get what is needed: four zero vector states, and one  latest decoded hidden state.
Again decoder function will be called. A newer decoder hidden state will be calculated.
It will be stored at index j (which is 6). J will be increased by 1. J will be 7. And next iteration will start.
Now when the last K (5) hidden states will be accessed using hidden_residuals_stack[j-K:j],
we will be getting values from index j-K (7-5 = 2) to j-1 (7-1 = 6)
So we will get the two latest decoder hidden states stored at index 5 and 6, along side other 0 vectors in other indices.
In this way we can access last 5 hidden states when there are more than or equal to 5 hidden states.
To follow this logic I added the decoded hidden state to j.
NOTE: I might have made a mistake. The initial decoder hidden state is not zero, but encoded_hidden[0].
So I think that should be included at index j-1 of hidden_residuals_d.

here argmax finds the index having high value so the output would be a single int value as there are no axis given.
say argmax is 8
but next, y = tf_embdlimit[8]

yes

but this embdlimit had the vector representation of words in glove. ie say - "the" and its corresponding vector.

yes

if we had to access the element, we could do embed_limit[vocab.index(word)] but also a word "the" is represented as row 0 in embed-limit.

Yes.

But here are we randomly selecting something from embd_limit based on y_index..

We are not randomly selecting anything.
The decoder first results in the probability distribution. Using argmax, the index of the maximum probability value is obtained. This index (y_index) corresponds to a word in vocabulary (which also corresponds to an embedding\word vector in embd_limit). Therefore the word_vector stored at y_index in embdlimit is the MOST PROBABLE word vector for that timestep as chosen by the decoder.
That is why y is set as tf_embd_limit[y_index]...since in decoder the previous word vector is treated as one of the input. So in the next timestep, y (tf_embd_limit[y_index]) becomes the one of the inputs to the decoder function.

but why i am confused is because , consider list of probailities [0.1 0.2 0.4...]
here 3 is argmax,  but why how do one know the order of embd_limit since it can be changed by say, sorting or ordering based on some condition. So in this cases, the index 3 of embd_limit can result in some other word.....but it may not be what was represented by 3 in y...
or am i just overcomplicating since the concept may actually be selecting whatever values in embdlimit based on the index in y

how do one know the order of embd_limit since it can be changed by say, sorting or ordering based on some condition.

But why would we do that?
If we randomly change order of the embd_limit at every training iteration, the program would never learn properly.

So in this cases, the index 3 of embd_limit can result in some other word.....but it may not be what was represented by 3 in y...
or am i just overcomplicating since the concept may actually be selecting whatever values in embdlimit based on the index in y? since there is no way of knowing what the index 3 contains since its a vector formed after lot of operations?

Let me explain.
Let's first talk about the connection between embd_limit and vocab_limit.
There is a special connection between the embeddings (word vectors) in embd_limit and the words in vocab_limit.
The embeddings are derived from the words following specific rules.
The word vector of 'lion' is meant to represent 'lion' only.
The word vectors also have interesting properties.
Like the vector representation of 'lion' may be found 'similar' (there are formulas to calculate similarities between vectors - check cosine similarity or euclidian distance) to vector representation of 'tiger'.
It has been sound that vector representations of word similar in meaning, are also similar to each other.
Now, if we treat the vector representation of 'lion'  as the vector representation for 'refrigerator', it will be totally wrong.
There is a connection among the word and its vector representation that needs to be appreciated.
We can't just treat any vector as a representation for any word. Actually we can, we can make a program like that where each word is randomly assigned with a unique vector but that would void the whole point of using algorithms like GloVe or word2vec. Embeddings made from these algos are not random but capture some meaning and context information of the word - which is why vector of lion may be found closer to the vector of tiger.
So there is a special connection between vocab_limit and embd_limit.
I made the arrays such that the if a word is stored at index i in vocab_limit then its vector representation will also be stored at index i in embd_limit.
Hope it's clear upto now.
Now, if you are seeking for this type of 'connection' between embd_limit and the probability distribution, you won't be finding any.
In fact, initially there is no actual connection between the list of probabilities and values of embd_limit.
The untrained network will be getting vector representations of word - it will encode them and then decode them into some set of float values that doesn't appear to mean anything.
But it can be treated to mean anything.  For example,, let us talk about a classification problem, say handwritten digit recognition. The classes are - 0,1,2,3,4,5,6,7,8, and 9.
Now the neural network may be made to output a 10 dimensional vector like:
[0.1 0.5 0.7 0.4 0.9 0.2 0.6 0.8 0.5 1]
(let's say this is the output before the softmax layer)
None of the values are yet officially probabilities before softmaxing (because they don't add up to 1) but for the sake of simplicity let me still call them probabilities.
So now, these output in itself really mean nothing.
But you can treat it such that value at index 0 (0.1) represents class 0 (0), value at index 1 represent class 1...and so on...
or you may treat it such that value at index 0 represents class 9, index 1 value reprsents class 5...and so on...
It doesn't really matter how you choose to treat the output as long as each probability value is made to uniquely represent one class/word vector.
It's same for this case.
The probability index 3 may be treated to represent value at index 3 of embd_limit, or may be some other rules of correspondence can be used.
It doesn't matter. So even if you sort or change the embedding values before starting the network, it wouldn't matter. However, you have to be consistent with whatever rule you choose (you have to stick with a specific way of treating the output throughout the training and prediction).
But why doesn't it matter?
Because the model with train and adjust its way based on how you treat the outputs.
For example, let us say, for a digit classification problem, the target output is 8 (class 8).
The neural network pre-softmax output is [0.1 0.5 0.7 0.4 0.9 0.2 0.6 0.5 0.99 0.8]
Now if you follow the rule that index i's value represents class i, then 0.99 here represents the chance of it being class 8 since 0.99 is in index 8 (starting from 0).
Looks like index 8 has the highest chance among the others and it will be chosen through argmax as the prediction of the network, SO the network will make a correct prediction. Loss will be lower. The network may just slighly increase the chance of class 8 if everything remains the same, through backpropagation.
So your network will adjust based on your treatment of the output in a way such that index i value represents class i probability.
But, what if you instead chose to treat the output such that index 2 (not 8) represents class 8, whereas index 8 represents class 2.
In that case, the probability of class 8 would be  0.7 (value of index 2 starting from 0).
The prediction would be class 2 represented by value at index 8 (highest value). So the prediction will be incorrect. The loss would be relatively higher too. The program will adjust its parameters through backpropagation so that its index 2 value is increased while other values are decreased under similar circumstances.
So. basically, the model will learn to adjust its parameters based on how you treat the connection between its raw output and the available classes or words in this case. So it doesn't exactly matter what precise rule of correspondence you follow. There is no 'inbuilt' connection between the output and the embeddings.

since there is no way of knowing what the index 3 contains since its a vector formed after lot of operations?

You can know what's in index 3 by using sess run to print the value at index 3 of the ndarray.

Thank you for the detailed explanation.

hidden = tf.multiply(og,tf.tanh(cell+RRA))
in the above code what exactly is the rra expected to do in the ideal case , since tanh would anyway convert it to a value between -1 to 1. cell is just a 500 dimensional matrix and rra is added to it, where rra is just a weighted sum of k window hidden states.
In the ideal case, is rra supposed to give more info for encoding so that more context is being available?

also during local attention, fixing d=10, is it a good tradeoff? since a lot of data would be eliminated as it wont't be of that size(21). In the ideal case, which would make it better, decreasing d or increasing d?

In the ideal case, is rra supposed to give more info for encoding so that more context is being available?

something like that.

would anyway convert it to a value between -1 to 1

Yes, but its value would still be influenced by RRA.

also during local attention, fixing d=10, is it a good tradeoff? since a lot of data would be eliminated as it wont't be of that size(21). In the ideal case, which would make it better, decreasing d or increasing d?

I am not sure what is supposed to be an 'ideal case' in this case.  The optimal value of d may depend on the task (translation? summarization?), the domain of the task (scientific articles? News? Social media posts?) and many other factors. Neither of the cases seem to be more ideal than the other.
There's no strict science to hyperparameter optimization; not yet. d is a hyperparameter. A lot of things in deep learning lack rigorous scientific\mathematical basis, and somewhat based on intuitions and stuffs.
You have to use some common sense and intuition to judge a good approx value range for d and then we may select one after some trial and error.
If you increase D too much, you may end up with lot of data which are less than the window size. There may be still ways to uses those data without eleminating them; but anyway, if D is too big, there won't be much point of using local attention over global attention.
If D is too small the decoder may not get enough encoded material to attend i.e it won't have enough context to make a good summary or whatever it is doing.
What you can do is check the average source text data, the target output data, and judge based on yourself, how much words from which position in the text are needed for sufficient context for each predicted word in summary. Use your human sense for summarization. If you find, say for example, you don't need to check more than the context provided by any 7 words in the source text to predict each output words in the target summary, chances are the machine should be able to work with 7 words sized window too. If it doesn't bring good results, you may increase d a bit..and so on. A bit trial and error is usually involved.

def np_nearest_neighbour(x):

xdoty = np.multiply(embedding,x)
xdoty = np.sum(xdoty,1)
xlen = np.square(x)
xlen = np.sum(xlen,0)
xlen = np.sqrt(xlen)
ylen = np.square(embedding)
ylen = np.sum(ylen,1)
ylen = np.sqrt(ylen)
xlenylen = np.multiply(xlen,ylen)
cosine_similarities = np.divide(xdoty,xlenylen)

which formula is this based on? also

def transform_out(output_text):
output_len = len(output_text)
transformed_output = np.zeros([output_len],dtype=np.int32)
for i in xrange(0,output_len):
transformed_output[i] = vocab_limit.index(vec2word(output_text[i]))
return transformed_output

how exactly does this make it logitc compatible?
is there any padding done at all in the code?
also you have made  target output length has to be always made equal to actual summary length rit? so whatever happens output has to have the same number of elements as in actual?

which formula is this based on?

Cosine similarity.
https://en.wikipedia.org/wiki/Cosine_similarity

is there any padding done at all in the code?

Don't remember. Probably not; since padding is mostly required for batching.

also you have made target output length has to be always made equal to actual summary length rit? so whatever happens output has to have the same number of elements as in actual?

I fed the target output length to the model, so that the decoder runs target_output_length times i.,e the summary length ends up the same as target output length.
Not sure what you mean by 'actual summary'. The target output is the 'actual summary' here in a sense. The model output is the predicted summary.

how exactly does this make it logitc compatible?

Input to transform_out is a list of word vectors representing the vectorized vesrion of the target summary sentence.
tf.sparse_softmax_cross_entropy_with_logits() requires the list of label no, of the correct class (in this case it can be index numbers chosen to uniquely represent the target summary word vectors).
Let's say there are three words in vocabulary: "Tiger, lion, eats"
Now tiger can be represented by its index 0, lion can be represented by its index 1, and eats can be represnted by its index 2.
Let's say the target summary is vector version of "Tiger eats"
One hot encoded version of it will be "[1 0 0 ] [0 0 1]"
And the format compatible with  tf.sparse_softmax_cross_entropy_with_logits() is [ 0 2 ].
(tiger being represented by its index no. 0, and eats being represented by its index no. 2; also these numbers represents the position of  '1' in the one hot encoded version of the same data)
So Transform_out will convert vector version of "Tiger eats" to [ 0 2 ] by simply finding the indices of the word vectors. This is only done for comparison with the probability distribution (loss calculation).

For batching u talked about the need for padding. But here as it is sample by sample, how does it eliminate the need for padding? also, why are certain summary lengths removed since how much ever is the actual summary length, the predicted summary output length also will be of same size.....ie len(train_out)
here is there complication between target output alignment vs the predicted output.

For batching u talked about the need for padding. But here as it is sample by sample, how does it eliminate the need for padding?

In batching you will be training multiple samples at once.
So multiple sentences will be represented in a nd-array and with linear-algebras they will be trained pretty much parallely.
However, sentences don't come with same length.
Let's consider size 2 batching (batching of only 2 samples):
sentence 1: Lion eats meat
sentence 2: lion and tigers are carnivores
Now how can we represent them in a tensor form if they aren't the same size.
In an nd array, each sentence would form a row....but you can't have tensor with rows of different size (similar to how a matrix can't have rows of different sizes).
So padding is done make the sizes equal.
In this case
sentence 1 can be changed to: lion eats meat
Now both are of equal length, and batch operation can be easily done.
No such need is there when dealing with one sample at a time.

Did u remove long summaries only to reduce training load?

Yes. It's written in the readme.

here is there complication between target output alignment vs the predicted output. or is it direct enough? Where target output is used only for length , and for loss calculation?

I am not sure I understand what you mean here exactly.
In this program there isn't much complication - the alignment is direct enough for comparison.

output = model(tf_text,tf_seq_len,tf_output_len)

#OPTIMIZER
cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=output,
labels=tf_summary))
is the ouput in here have any relation with output in this

def attention_decoder_body(i,j,decoded_hidden,cell_d,hidden_residuals_d,output):

here output is having probaility distribution, in this decoder part rit? but what about in the output associated with model function?
what is the program flow? does it start with values fed here

_,loss,pred = sess.run([optimizer,cost,prediction],feed_dict={tf_text: train_texts[i],
tf_seq_len: len(train_texts[i]),
tf_summary: train_out,
tf_output_len: len(train_out)})

then, when does it call model, which would essentially calculate everything.

def attention_decoder_body(i,j,decoded_hidden,cell_d,hidden_residuals_d,output):
    #LOCAL ATTENTION

    G,pt = align(encoded_hidden,decoded_hidden,Wp,Vp,Wa,tf_seq_len)
    local_encoded_hidden = encoded_hidden[pt-D:pt+D+1]
    weighted_encoded_hidden = tf.multiply(local_encoded_hidden,G)
    context_vector = tf.reduce_sum(weighted_encoded_hidden,0)
    context_vector = tf.reshape(context_vector,[1,2*hidden_size])

    attended_hidden = tf.tanh(tf.matmul(tf.concat([context_vector,decoded_hidden],1),Wc))

    #DECODER

    **y = tf.matmul(attended_hidden,Ws)**

The probability distribution is formed in the bolded part.
It is then stored in output tensorarray

output = output.write(i,tf.reshape(y,[vocab_len]))

Near the end of model function, using stack function of tensorarray, the values of tensorarray output is retrieved to form a Tensor output

output = output.stack()

The tensor output now contains the probability distributions. It is returned at the end of model function.

return output

The output (probability distributions) returned by the model function is used for cost calculation.
(as you can see the model function is also called here).

output = model(tf_text,tf_seq_len,tf_output_len)
#OPTIMIZER
cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=output, labels=tf_summary))

also what is the use of display_step ?

Display step can be used to display the training results only at certain intervals.
It's usually more useful to use it to calculate validation loss & accuracy and show it only at certain intervals.
Calculating validation loss and accuracy at each training step can be expensive.
It has no use here. Display step is set to 1, it means all predicted data will be shown.

encoded_hidden = tf.concat([hidden_forward,hidden_backward],1)
is there any way to print the value of this tensorarray...will simply printing it inside the sess.run work?

sess.run([optimizer,cost,prediction],feed_dict={tf_text: train_texts[i],
tf_seq_len: len(train_texts[i]),
tf_summary: train_out,
tf_output_len: len(train_out)})

is there any way to print the value of this tensorarray encodedhidden ...will simply printing it inside the sess.run work?

Probably not. But you can easily use stack() to convert into a tensor and then display it using sess.run. That way you can print the values of encoded hidden. The values of encoded hidden are already stacked within a Tensor form the tensorarray, so you don't have to do anything extra.

IS THERE ANY WAY TO KNOW HOW MANY LSTM units or hidden layers(500 is the dimension rit?) we are using?

Forward encoder - 1 LSTM
Backward encoder - 1 LSTM
Decoder - 1 LSTM
a total of 3 if I am not missing something.

What is the activation function used here?

Used where?
Activation functions (sigmoid, tanh) are spread throughout the LSTM equations.

Probably not. But you can easily use stack() to convert into a tensor and then display it using sess.run. That way you can print the values of encoded hidden. The values of encoded hidden are already stacked within a Tensor form the tensorarray, so you don't have to do anything extra.

when i try running the function through sess.run, it is requireed to specify arguments and everything. But the tensorarray is local to the function. So how to access something inside a function? Even in model(), encoded_hidden(concatenated array)  can't be accessed.

when i try running the function through sess.run, it is requireed to specify arguments and everything. But the tensorarray is local to the function. So how to access something inside a function? Even in model(), encoded_hidden(ie concatenated array) can't be accessed.

You have to return it from the function. Store it in a global variable then use sess.run.
For example encoded_hidden has to be returned from model().
Use "return output, encoded_hidden" at the end of model().
Then use "output, encoded_hidden = model(tf_text,tf_seq_len,tf_output_len)" instead of "output = model(tf_text,tf_seq_len,tf_output_len)"
After that, you should be able to access it.

It can restore the saved model parameters and continue training from where previously left without starting training all over again.
You have to write the code for loading the saved model, though.


Update valid connection checkers and exceptions sorters

Update valid connection checkers and exceptions sorters for PostgreSQL, Oracle and MS SQL Server datasources
This fixes #32
This fixes #33
This fixes #34

@jmesnil , the null checkers are expected to do nothing. I am not sure we want the default to have an effect. An application must be implemented specifically to handle thrown exception by checkers. The null checkers and sorters being the default seems a valid default value.

As discussed, we want default checkers to be effective.

@jfdenise I tested locally with a snapshot of EAP 7.4 and the tests passed after I updated them.


Is CommonJS module loading supported for addons?

Is this the correct way to require addons when using the CommonJS approach?
var Dexie = require("dexie");
var DexieObservable = require("dexie/addons/Dexie.Observable/Dexie.Observable");
var DexieSyncable = require("dexie/addons/Dexie.Syncable/Dexie.Syncable");

I'm currently using browserify, and get an error when loading the .js it generates.
Uncaught Error: Cannot find module 'Dexie'
It looks like it's properly detecting that it should use the CommonJS path, but I think the dependency names aren't being resolved.  Hopefully the fix is obvious, I'm just getting my feet wet with this stuff and I've been trying to figure it out pretty much all day.
Keep up the good work!

Do you still have the problem in version 1.1? Might be issues with the dots in the file name. I'll spend some time with this later on to see what might be the error.

+1 for same question, directory "addons" was added in ".npmignore" so all addon files are not included in npm package.

Thanks for the pullrequest!
Need to make a new release in order to publish to npm though. Hopefully tomorrow.


Allow specifying default filters on the web interface

It would be great if a default set of filters can be set when opening the interface.
If there are a lot of pods, there could be various urls which all of them have, but are not useful when debugging.
An example would be something like request.path != "/api/healthcheck".

Hi @zhang-li-80
Thanks for the feature idea.
I can suggest two approaches that might be useful for you:

there is option to specify user agents that you want to ignore (health checks are most likely coming from some specific user agent that you can ignore and they will not be inserted at all)
the way to configure it is using the tap.ignored-user-agents so if you are running mizu tap you can override it like this
./mizu tap --set tap.ignored-user-agents=kube-probe [it is array of user agents that will be filtered if the user-agent header includes one of the values [in case you want to disable multiple user agents ``````./mizu tap --set tap.ignored-user-agents=kube-probe --set tap.ignored-user-agents=monitoring```]

the other way to disable insertion of some data that you don't want is to use the insertion filter, so you can configure filter (same syntax in the web ui) and only "passed" items will be inserted..  the way to use this feature -set tap.insertion-filter=http [this will insert only http body] and you can also make it more readable if you want to create file contains your filter and pass the path to the file as value, e.g: -set tap.insertion-filter=fiilter-file.bfl

I hope it helps and you are more than welcome to suggest more improvements.

Thanks for the quick reply.
It looks like the tap.ignored-user-agents works for the health checks.

I have both of them set in the config file.

Thanks for letting us know.
I will investigate and check that this option is not broken and try to update here.
Can you just tell me which version are you using?

I had a chance to run it myself and I see it is working (in version 29).
the issue might be related to the value you specified in the filter, the easiest way to be sure the filter is correct is to write it inside file and pass this file so you will not have issue with prefix specific characters (like ") [this is at least what happen to me) because the filter need to be dst.name != "my-service.k8s"..
if you can check if via file (e.g create file filter.bfl and write the exact same filter that you wrote and let me know if it solves your problem..
Hope it will and you will be able to start using it - if no, you are more than welcome to write here and I will try to help more

Running version 29.0 (main) as well.
The tap.insertion-filter config issue is my fault.
Found out that the config file wasn't saved
Both the config and the filter.bfl file work.


Make OpenApp a write command

cc @iwiznia @tgolen @roryabraham @marcochavezf
Details

This PR makes OpenApp a write command so that for anonymous users we make sure to process OpenApp first before any requests that are made after it.
Summary of my findings from this thread

The first request an anonymous user makes is OpenReport of the parent room, OpenReport is processed first and does not return a lastUpdateID
The app then makes a call to OpenApp as usual, but its response is not processed immediately as it’s a read command, the sequential queue must process write commands first, and in this case it’s still processing OpenReport
Anon clicks quickly on a thread and triggers another call to OpenReport which we will process before OpenApp ’s response
Once we attempt to save the response of the 2ndOpenReport , the sequential queue finds out we don’t have a lastUpdateID set here and calls ReconnectApp
The sequential queue is paused until we process ReconnectApp which comes back with a lastUpdateID equal to OpenReport ’s
We unpause the queue to apply the update from OpenReport but since it has the same update ID we now have in the client we don’t apply its response and we end up with the component not having a report set.

So in conclusion, I think this is due to navigating to the thread quickly before OpenApp has finished, so if we make it a write, it’ll be processed first before we process  OpenReport when opening the thread

Fixed Issues

$ https://github.com/Expensify/App/issues/36430
Tests

[ ] Verify that no errors appear in the JS console

To reproduce locally add a sleep(3) to openApp here, as to reproduce this bug OpenReport needs to finish before OpenApp.
Create a workspace room with visibility set to public, create a thread inside the room.
Open the room as an anonymous user in an incognito window, and click on the thread as fast as you can.
Make sure it loads properly.

Offline tests

QA Steps

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[X] I linked the correct issue in the ### Fixed Issues section above
[X] I wrote clear testing steps that cover the changes made in this PR

[X] I added steps for local testing in the Tests section
[X] I added steps for the expected offline behavior in the Offline steps section
[X] I added steps for Staging and/or Production testing in the QA steps section
[X] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[X] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[X] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[X] I included screenshots or videos for tests on all platforms
[X] I ran the tests on all platforms & verified they passed on:

[X] Android / native
[X] Android / Chrome
[X] iOS / native
[X] iOS / Safari
[X] MacOS / Chrome / Safari
[X] MacOS / Desktop

[X] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[X] I followed proper code patterns (see Reviewing the code)

[X] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[X] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[X] I verified that comments were added to code that is not self explanatory
[X] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[X] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[X] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[X] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[X] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[X] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[X] I verified the JSDocs style guidelines (in STYLE.md) were followed

[X] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[X] I followed the guidelines as stated in the Review Guidelines
[X] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[X] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[X] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[X] I verified that if a function's arguments changed that all usages have also been updated correctly
[X] If a new component is created I verified that:

[X] A similar component doesn't exist in the codebase
[X] All props are defined accurately and each prop has a /** comment above it */
[X] The file is named correctly
[X] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[X] The only data being stored in the state is data necessary for rendering and nothing else
[X] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[X] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[X] All JSX used for rendering exists in the render method
[X] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[X] If any new file was added I verified that:

[X] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[X] If a new CSS style is added I verified that:

[X] A similar style doesn't already exist
[X] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[X] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[X] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[X] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[X] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[X] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[X] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/9680864/dafc9c93-f2f9-4ed2-8914-e40da05007b2

MacOS: Desktop

Seems we'd need to update the unit test cases and types too

Yeah I'm looking into it now

@tgolen I agree and I thought about that, but OpenApp can actually be a write command as it creates a chat report here as @marcochavezf pointed out in the thread. Also, in Web-E openApp eventually calls reconnectApp, which is a write command, so I'm not really sure why we're having it as a read command.

OK, thanks! That's great if it results in WRITEs being done.

Web-E openApp eventually calls reconnectApp, which is a write command

reconnectApp is a write command??? What does it write?

reconnectApp is a write command??? What does it write?

I'm not sure, if it doesn't actually write anything, it seems like we made it solely to have it block the queue? cc @tgolen I can see it was added here

The design doc said:
"It will call API.write (doing this so that it blocks all other calls)"
So yeah, I think your assumption is correct. No wonder why I didn't like
manipulating read/write like that!
On Thu, Feb 22, 2024 at 1:08 PM Youssef Lourayad @.***>
wrote:

reconnectApp is a write command??? What does it write?
I'm not sure, if it doesn't actually write anything, it seems like we made
it solely to have it block the queue? cc @tgolen
https://github.com/tgolen I can see it was added here
https://github.com/Expensify/App/pull/23516
—
Reply to this email directly, view it on GitHub
https://github.com/Expensify/App/pull/37062#issuecomment-1960207408, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJMAB2ECJFK2PIMR3WPSCDYU6QTLAVCNFSM6AAAAABDUEGEPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRQGIYDONBQHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

cc'ing @marcaaron as well

Looks like we just need a checklist, since this is hard to reproduce on mobile, I think we can just do web screenshots.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/marcochavezf in version: 1.4.45-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/puneetlath in version: 1.4.45-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Use <IP_ADDRESS> as account.endpoint_suffix in tests.

This way we comply with production code requirement that account must be provided (will be resolved to 127) and also provide a valid host that can be used for starting stub server locally.
I also tried to use custom DNSResolver (https://stackoverflow.com/a/73358765).
That can help to start stub server on "abc.localhost" but there is no guarantee that Native calls in OpenDAL will use it to resolve host name looking like "abc.localhost", I think faking account name is less invasive and simpler.

Unrelated
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to ..
retest this please


check ITM for temporary storage location

We should check the usage of temporary files in ITM.
The location might be switched to fargate ephemeral storage if it is not already.

Closing, as we have migrated nearly all job types from the ITM to worker nodes.


New policy: Namespace labels propagator

Create a context-aware mutating policy that operates against Namespaced resources.
Whenever a resource is created/updated the policy ensures a list of user-specified labels are copied from the Namespace to the object. Labels defined on the Namespace have higher precedence over the ones already defined inside of the namespaced resource.
The configuration of the policy will be something like that:
propagated-labels:
- cost-center
- field.cattle.io/projectId

In this scenario, when a Pod is created, the policy would ensure the keys cost-center and field.cattle.io/projectId are copied from the Namespace object into the Pod itself.
The policy will propagate the desired labels only if they are already set on the Namespace.
Known caveats
Labels are not kept in sync automatically
The policy will propagate the labels only when the item is created/updated. A change to the key value inside of the Namespace will not be automatically propagated.
For example, let's assume the policy is configured to propagate the label greeting and the Namespace test has this key set to hello world.
A new Pod named nginx is created under the test Namespace, the policy will add the greeting label with value hello world. The Namespace is edited, the greetings label has the a new value ciao mondo. The nginx Pod is not updated, it still has a label greetings with value hello world.
The label will be synchronized only if the nginx Pod is updated.
Labels are not removed
Let's assume the policy is configured to propagate the label greeting and the Namespace test has this key set to hello world.
A new Pod named nginx is created under the test Namespace, the policy will add the greeting label with value hello world.
The Namespace is edited, the greetings label is removed. No update is done to the nginx Pod. Moreover, if the nginx Pod is edited, the policy will be triggered again but will not remove the greetings policy from it.

@Martin-Weiss we would like to start working on this policy. Can you take a quick look at this issue and tell us if this is what you were looking for? Also, please take a careful look at the "limitations" section