Failing to get activations from fine-tuned resnet model

I have a fine-tuned resnet50 model which i I'm trying to visualize the activations, although I'm facing a few issues when calling get_activations function. Code and output below:
Building model fn:
def load_and_configure_model(model_name, optimizer, loss, metrics, path):
  model = 0
  num_classes = 250
  if model_name == 'resnet50':
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    conv_model = Model(inputs=model.input,outputs=transfer_layer.output)
    conv_model.trainable = True
    for layer in conv_model.layers:
      layer.trainable = False
    for layer in conv_model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(conv_model)
    transferred_resnet50.add(Dense(num_classes, activation='softmax'))
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    model = transferred_resnet50
  if not path == None :
    model.load_weights(path)
  return model

Loading model & img
weight_file = DRIVE_DIR+'ImageCLEF2013PlantTask/Models/ResNet50/crop_and_translate/resnet_50.h5'
loss = tf.keras.losses.sparse_categorical_crossentropy

metrics = ['accuracy']
optimizer = Adam(lr=2e-6)

resnet_50_crop_and_translate = load_and_configure_model('resnet50', optimizer, loss, metrics, weight_file)
resnet_50_crop_and_translate.compile(optimizer=optimizer,loss=loss)

img = cv2.imread(DRIVE_DIR + DATA_DIR+'/visualization/test_image.jpg',cv2.IMREAD_UNCHANGED)
res = cv2.resize(img, dsize=(224, 224))

activations = keract.get_activations(resnet_50_crop_and_translate, np.expand_dims(res,axis=0), nodes_to_evaluate=None, output_format='full', nested=False, auto_compile=False)

First Error, although warnings concerning eager execution and other related exceptions seemed also to be thrown:
Run it without eager mode. Paste those commands at the beginning of your script:
> import tensorflow as tf
> tf.compat.v1.disable_eager_execution()
Run it without eager mode. Paste those commands at the beginning of your script:
> import tensorflow as tf
> tf.compat.v1.disable_eager_execution()

ValueError: Tensor Tensor("avg_pool_1/Mean:0", shape=(None, 2048), dtype=float32) is not an element of this graph.

Then when I have also tried:
import keras.backend as K
K.clear_session()
activations = keract.get_activations(resnet_50_crop_and_translate, np.expand_dims(res,axis=0), nodes_to_evaluate=None, output_format='full', nested=False, auto_compile=False)

Which raised:
---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)

<ipython-input-17-cb7cc7958838> in <module>()
----> 1 activations = keract.get_activations(resnet_50_crop_and_translate, np.expand_dims(res,axis=0),layer_names=['conv5_block3_out'], nodes_to_evaluate=None, output_format='full', nested=False, auto_compile=False)

/usr/local/lib/python3.6/dist-packages/keract/keract.py in get_activations(model, x, layer_names, nodes_to_evaluate, output_format, nested, auto_compile)
    319             network_layers = ', '.join([layer.name for layer in model.layers])
    320             raise KeyError('Could not find a layer with name: [{}]. '
--> 321                            'Network layers are [{}]'.format(', '.join(layer_names), network_layers))
    322         else:
    323             raise ValueError('Nodes list is empty. Or maybe the model is empty.')

KeyError: 'Could not find a layer with name: [conv5_block3_out]. Network layers are [model_1, dense_1]'

Full trace from error no. 1 (without specifying layer names):

Run it without eager mode. Paste those commands at the beginning of your script:
> import tensorflow as tf
> tf.compat.v1.disable_eager_execution()
Run it without eager mode. Paste those commands at the beginning of your script:
> import tensorflow as tf
> tf.compat.v1.disable_eager_execution()

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/usr/local/lib/python3.6/dist-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
    102     try:
--> 103         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    104     except Exception:

18 frames

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     99             print('> tf.compat.v1.disable_eager_execution()')
--> 100             raise e
    101

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     90         try:
---> 91             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     92         except AttributeError:  # one way to avoid forcing non eager mode.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in function(inputs, outputs, updates, name, **kwargs)
   4086   return GraphExecutionFunction(
-> 4087       inputs, outputs, updates=updates, name=name, **kwargs)
   4088

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __init__(self, inputs, outputs, updates, name, **session_kwargs)
   3809     # Index 0 = total loss or model output for `predict`.
-> 3810     with ops.control_dependencies([self.outputs[0]]):
   3811       updates_ops = []

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in control_dependencies(control_inputs)
   5358   else:
-> 5359     return get_default_graph().control_dependencies(control_inputs)
   5360

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in control_dependencies(self, control_inputs)
   4814         c = c.op
-> 4815       c = self.as_graph_element(c)
   4816       if isinstance(c, Tensor):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in as_graph_element(self, obj, allow_tensor, allow_operation)
   3725     with self._lock:
-> 3726       return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
   3727

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _as_graph_element_locked(self, obj, allow_tensor, allow_operation)
   3804       if obj.graph is not self:
-> 3805         raise ValueError("Tensor %s is not an element of this graph." % obj)
   3806       return obj

ValueError: Tensor Tensor("avg_pool_1/Mean:0", shape=(None, 2048), dtype=float32) is not an element of this graph.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)

<ipython-input-18-1fc67112db22> in <module>()
----> 1 activations = keract.get_activations(resnet_50_crop_and_translate, np.expand_dims(res,axis=0), nodes_to_evaluate=None, output_format='full', nested=False, auto_compile=False)

/usr/local/lib/python3.6/dist-packages/keract/keract.py in get_activations(model, x, layer_names, nodes_to_evaluate, output_format, nested, auto_compile)
    347
    348     if len(layer_outputs) > 0:
--> 349         activations = _evaluate(model, layer_outputs.values(), x, y=None, auto_compile=auto_compile)
    350     else:
    351         activations = {}

/usr/local/lib/python3.6/dist-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
    114                         acts.append(n.numpy())
    115             return acts
--> 116         return eval_fn(model._feed_inputs)
    117
    118

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     98             print('> import tensorflow as tf')
     99             print('> tf.compat.v1.disable_eager_execution()')
--> 100             raise e
    101
    102     try:

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     89     def eval_fn(k_inputs):
     90         try:
---> 91             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     92         except AttributeError:  # one way to avoid forcing non eager mode.
     93             if y is None:  # tf 2.3.0 upgrade compatibility.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in function(inputs, outputs, updates, name, **kwargs)
   4085         raise ValueError(msg)
   4086   return GraphExecutionFunction(
-> 4087       inputs, outputs, updates=updates, name=name, **kwargs)
   4088
   4089

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __init__(self, inputs, outputs, updates, name, **session_kwargs)
   3808     # dependencies in call.
   3809     # Index 0 = total loss or model output for `predict`.
-> 3810     with ops.control_dependencies([self.outputs[0]]):
   3811       updates_ops = []
   3812       for update in updates:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in control_dependencies(control_inputs)
   5357     return NullContextmanager()
   5358   else:
-> 5359     return get_default_graph().control_dependencies(control_inputs)
   5360
   5361

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in control_dependencies(self, control_inputs)
   4813           (hasattr(c, "_handle") and hasattr(c, "op"))):
   4814         c = c.op
-> 4815       c = self.as_graph_element(c)
   4816       if isinstance(c, Tensor):
   4817         c = c.op

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in as_graph_element(self, obj, allow_tensor, allow_operation)
   3724
   3725     with self._lock:
-> 3726       return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
   3727
   3728   def _as_graph_element_locked(self, obj, allow_tensor, allow_operation):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _as_graph_element_locked(self, obj, allow_tensor, allow_operation)
   3803       # Actually obj is just the object it's referring to.
   3804       if obj.graph is not self:
-> 3805         raise ValueError("Tensor %s is not an element of this graph." % obj)
   3806       return obj
   3807     elif isinstance(obj, Operation) and allow_operation:

ValueError: Tensor Tensor("avg_pool_1/Mean:0", shape=(None, 2048), dtype=float32) is not an element of this graph.

Try to structure your code in this way instead:
import numpy as np
from tensorflow.keras.applications.resnet import ResNet50
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model

import keract

def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50()
    for layer in model.layers:
        layer.trainable = False
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')(transfer_layer.output)
    model = Model(inputs=model.input, outputs=x)
    model.trainable = True
    model.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        model.load_weights(path)
    return model

def main():
    resnet_50_crop_and_translate = load_and_configure_model(
        'resnet50',
        'adam',
        'categorical_crossentropy',
        ['accuracy'],
        None
    )

    res = np.ones(shape=(1, 224, 224, 3))
    activations = keract.get_activations(
        resnet_50_crop_and_translate, res,
        output_format='full',
        auto_compile=False
    )
    print(activations.keys())

if __name__ == '__main__':
    main()

The nested models can sometimes return weird errors.

Try to structure your code in this way instead:
import numpy as np
from tensorflow.keras.applications.resnet import ResNet50
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model

import keract

def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50()
    for layer in model.layers:
        layer.trainable = False
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')(transfer_layer.output)
    model = Model(inputs=model.input, outputs=x)
    model.trainable = True
    model.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        model.load_weights(path)
    return model

def main():
    resnet_50_crop_and_translate = load_and_configure_model(
        'resnet50',
        'adam',
        'categorical_crossentropy',
        ['accuracy'],
        None
    )

    res = np.ones(shape=(1, 224, 224, 3))
    activations = keract.get_activations(
        resnet_50_crop_and_translate, res,
        output_format='full',
        auto_compile=False
    )
    print(activations.keys())

if __name__ == '__main__':
    main()

The nested models can sometimes return weird errors.

You can also follow some of the examples in: https://github.com/philipperemy/keract/blob/20846aaed0da68d5c328864b56d9a07fd696a76c/tests/get_activations_test.py#L16

You can also follow some of the examples in: https://github.com/philipperemy/keract/blob/20846aaed0da68d5c328864b56d9a07fd696a76c/tests/get_activations_test.py#L16

Worked out nicely, thank you very much for the attention and patience.

Worked out nicely, thank you very much for the attention and patience.

@FalsoMoralista great! Ideally your case should be properly handled but it's quite a pain to handle all the cases.

@FalsoMoralista great! Ideally your case should be properly handled but it's quite a pain to handle all the cases.

[Edit] I just realized after trying some predictions that i've copied/pasted the code withouth trying to load the weights, when i do it using your code the following error show up: ValueError: You are trying to load a weight file containing 2 layers into a model with 107 layers. I'm trying to figure out why, maybe a keras bug.

[Edit] I just realized after trying some predictions that i've copied/pasted the code withouth trying to load the weights, when i do it using your code the following error show up: ValueError: You are trying to load a weight file containing 2 layers into a model with 107 layers. I'm trying to figure out why, maybe a keras bug.

Fixed by doing:
def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        transferred_resnet50.load_weights(path)
    return model

Fixed by doing:
def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        transferred_resnet50.load_weights(path)
    return model

Great! Keras can sometimes be a pain to use.

Great! Keras can sometimes be a pain to use.

Nevermind what i said earlier, i realized that i was returning the model before loading the weights.

Fixed by doing:
def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        transferred_resnet50.load_weights(path)
    return model

When in fact after trying to get the activations from the fine-tuned model leads to the same error:
Python Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]

Nevermind what i said earlier, i realized that i was returning the model before loading the weights.

Fixed by doing:
def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        transferred_resnet50.load_weights(path)
    return model

When in fact after trying to get the activations from the fine-tuned model leads to the same error:
Python Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]

Tryied downgrading tf to 2.3 and 2.1, unsucessfully.
import numpy as np
from tensorflow.keras.applications.resnet import ResNet50
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model, Sequential

import keract

def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        transferred_resnet50.load_weights(path)
        #transferred_resnet50.build(input_shape=(1, 224, 224, 3))
    return transferred_resnet50

weight_file = DRIVE_DIR+'ImageCLEF2013PlantTask/Models/ResNet50/crop_and_translate/resnet_50.h5'

resnet_50_crop_and_translate = load_and_configure_model(
  'resnet50',
  'adam',
  'categorical_crossentropy',
  ['accuracy'],
  weight_file
)

img = im.open(DRIVE_DIR + DATA_DIR+'/visualization/test_class_208/18960.jpg')
img = img.resize((224,224),im.ANTIALIAS)
plt.imshow(img)
plt.show()

img = np.expand_dims(img,axis=0)
#tf.keras.backend.clear_session()

activations = keract.get_activations(
model=resnet_50_crop_and_translate, x=img,
output_format='full',
auto_compile=False
    )
print(activations.keys())

Complete traceback
---------------------------------------------------------------------------

InvalidArgumentError                      Traceback (most recent call last)

/usr/local/lib/python3.6/dist-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
     99     try:
--> 100         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    101     except Exception:

8 frames

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     87         try:
---> 88             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     89         except AttributeError:  # one way to avoid forcing non eager mode.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3824     fetched = self._callable_fn(*array_vals,
-> 3825                                 run_metadata=self.run_metadata)
   3826     self._call_fetch_callbacks(fetched[-len(self._fetches):])

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1471                                                self._handle, args,
-> 1472                                                run_metadata_ptr)
   1473         if run_metadata:

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
	 [[dense/Softmax/_3215]]
  (1) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)

<ipython-input-8-2bd3d4432caa> in <module>()
     50 resnet_50_crop_and_translate, img,
     51 output_format='full',
---> 52 auto_compile=False
     53     )
     54 print(activations.keys())

/usr/local/lib/python3.6/dist-packages/keract/keract.py in get_activations(model, x, layer_names, nodes_to_evaluate, output_format, nested, auto_compile)
    316
    317     if len(layer_outputs) > 0:
--> 318         activations = _evaluate(model, layer_outputs.values(), x, y=None, auto_compile=auto_compile)
    319     else:
    320         activations = {}

/usr/local/lib/python3.6/dist-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
    100         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    101     except Exception:
--> 102         return eval_fn(model._feed_inputs)
    103
    104

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     86     def eval_fn(k_inputs):
     87         try:
---> 88             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     89         except AttributeError:  # one way to avoid forcing non eager mode.
     90             if y is None:  # tf 2.3.0 upgrade compatibility.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3823
   3824     fetched = self._callable_fn(*array_vals,
-> 3825                                 run_metadata=self.run_metadata)
   3826     self._call_fetch_callbacks(fetched[-len(self._fetches):])
   3827     output_structure = nest.pack_sequence_as(

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1470         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1471                                                self._handle, args,
-> 1472                                                run_metadata_ptr)
   1473         if run_metadata:
   1474           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
	 [[dense/Softmax/_3223]]
  (1) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
0 successful operations.
0 derived errors ignored.

Tryied downgrading tf to 2.3 and 2.1, unsucessfully.
import numpy as np
from tensorflow.keras.applications.resnet import ResNet50
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model, Sequential

import keract

def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    transferred_resnet50.compile(optimizer=optimizer, loss=loss, metrics=metrics)
    if path is not None:
        transferred_resnet50.load_weights(path)
        #transferred_resnet50.build(input_shape=(1, 224, 224, 3))
    return transferred_resnet50

weight_file = DRIVE_DIR+'ImageCLEF2013PlantTask/Models/ResNet50/crop_and_translate/resnet_50.h5'

resnet_50_crop_and_translate = load_and_configure_model(
  'resnet50',
  'adam',
  'categorical_crossentropy',
  ['accuracy'],
  weight_file
)

img = im.open(DRIVE_DIR + DATA_DIR+'/visualization/test_class_208/18960.jpg')
img = img.resize((224,224),im.ANTIALIAS)
plt.imshow(img)
plt.show()

img = np.expand_dims(img,axis=0)
#tf.keras.backend.clear_session()

activations = keract.get_activations(
model=resnet_50_crop_and_translate, x=img,
output_format='full',
auto_compile=False
    )
print(activations.keys())

Complete traceback
---------------------------------------------------------------------------

InvalidArgumentError                      Traceback (most recent call last)

/usr/local/lib/python3.6/dist-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
     99     try:
--> 100         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    101     except Exception:

8 frames

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     87         try:
---> 88             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     89         except AttributeError:  # one way to avoid forcing non eager mode.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3824     fetched = self._callable_fn(*array_vals,
-> 3825                                 run_metadata=self.run_metadata)
   3826     self._call_fetch_callbacks(fetched[-len(self._fetches):])

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1471                                                self._handle, args,
-> 1472                                                run_metadata_ptr)
   1473         if run_metadata:

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
	 [[dense/Softmax/_3215]]
  (1) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)

<ipython-input-8-2bd3d4432caa> in <module>()
     50 resnet_50_crop_and_translate, img,
     51 output_format='full',
---> 52 auto_compile=False
     53     )
     54 print(activations.keys())

/usr/local/lib/python3.6/dist-packages/keract/keract.py in get_activations(model, x, layer_names, nodes_to_evaluate, output_format, nested, auto_compile)
    316
    317     if len(layer_outputs) > 0:
--> 318         activations = _evaluate(model, layer_outputs.values(), x, y=None, auto_compile=auto_compile)
    319     else:
    320         activations = {}

/usr/local/lib/python3.6/dist-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
    100         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    101     except Exception:
--> 102         return eval_fn(model._feed_inputs)
    103
    104

/usr/local/lib/python3.6/dist-packages/keract/keract.py in eval_fn(k_inputs)
     86     def eval_fn(k_inputs):
     87         try:
---> 88             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     89         except AttributeError:  # one way to avoid forcing non eager mode.
     90             if y is None:  # tf 2.3.0 upgrade compatibility.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3823
   3824     fetched = self._callable_fn(*array_vals,
-> 3825                                 run_metadata=self.run_metadata)
   3826     self._call_fetch_callbacks(fetched[-len(self._fetches):])
   3827     output_structure = nest.pack_sequence_as(

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1470         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1471                                                self._handle, args,
-> 1472                                                run_metadata_ptr)
   1473         if run_metadata:
   1474           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
	 [[dense/Softmax/_3223]]
  (1) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,224,224,3]
	 [[{{node input_1}}]]
0 successful operations.
0 derived errors ignored.

I'm getting the same error as @FalsoMoralista

@nandaw  I was able to manage by doing as follow:
First i had my model set up as normally so I was able to load the weights from my fine-tunned model (not sure if it is actually necessary for loading the weights). Then i created a submodel using the layers defined at "input" and "output" (the output is the last fine-tuned layer).
import numpy as np
from tensorflow.keras.applications.resnet import ResNet50
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model, Sequential

import keract

def load_and_configure_model(model_name, optimizer, loss, metrics, path):
    num_classes = 250
    assert model_name == 'resnet50'
    model = ResNet50(include_top=True, weights='imagenet')
    transfer_layer = model.get_layer('avg_pool')
    x = Dense(num_classes, activation='softmax')
    model = Model(inputs=model.input, outputs=transfer_layer.output)
    model.trainable = True
    for layer in model.layers:
      layer.trainable = False
    for layer in model.layers:
      trainable = ('conv5_block3' in layer.name)
      layer.trainable = trainable
    transferred_resnet50 = Sequential()
    transferred_resnet50.add(model)
    transferred_resnet50.add(x)
    if path is not None:
        transferred_resnet50.load_weights(path)
    input = transferred_resnet50.get_layer('model').input
    output = transferred_resnet50.get_layer('model').get_layer('conv5_block3_out').output
    model = Model(inputs=input, outputs=output)

    return model

Then basically all you have to do is to predict from the desired image and call the "display" method from keract.
 What I'm trying to do is do get the activation for each one of the slices:
def build_model():

    base_model = InceptionV3(weights='imagenet',
                                   include_top=False,
                                   input_shape=(IMAGE_SIZE, IMAGE_SIZE, 3))
    last = base_model.output

    x = GlobalAveragePooling2D()(last)

    model1 = Model(base_model.input, x)

    input_tensor = Input(shape=(N_SLICES, IMAGE_SIZE, IMAGE_SIZE, 3))
    timeDistributed_layer = TimeDistributed(model1)(input_tensor)

    #the ouput of the model is [1, 1, 2048], need to squeeze
    # som = Lambda(lambda x: K.squeeze(K.squeeze(x,2),2))(timeDistributed_layer)

    bnn = Bidirectional(LSTM(1000))(timeDistributed_layer)
    bnn = Dropout(0.5)(bnn)

    bnn = Dense(512, activation='relu')(bnn)
    bnn = Dropout(0.3)(bnn)

    preds = Dense(3, activation='softmax', name='out')(bnn)

    model = Model(input_tensor, preds)

    return model

And I got the following error:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~/tensorflow/lib/python3.6/site-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
     99     try:
--> 100         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    101     except Exception:

AttributeError: 'Model' object has no attribute '_feed_targets'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
~/tensorflow/lib/python3.6/site-packages/keract/keract.py in eval_fn(k_inputs)
     85         try:
---> 86             return K.function(k_inputs, nodes_to_evaluate)(model._standardize_user_data(x, y))
     87         except AttributeError:  # one way to avoid forcing non eager mode.

AttributeError: 'Model' object has no attribute '_standardize_user_data'

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-10-452c21376a85> in <module>
      1 # keract.get_activations(model_time, x, layer_names='lstm',nested=True)
----> 2 time_d = keract.get_activations(model, inputs,nested=True)

~/tensorflow/lib/python3.6/site-packages/keract/keract.py in get_activations(model, x, layer_names, nodes_to_evaluate, output_format, nested, auto_compile)
    340
    341     if len(layer_outputs) > 0:
--> 342         activations = _evaluate(model, layer_outputs.values(), x, y=None, auto_compile=auto_compile)
    343     else:
    344         activations = {}

~/tensorflow/lib/python3.6/site-packages/keract/keract.py in _evaluate(model, nodes_to_evaluate, x, y, auto_compile)
    100         return eval_fn(model._feed_inputs + model._feed_targets + model._feed_sample_weights)
    101     except Exception:
--> 102         return eval_fn(model._feed_inputs)
    103
    104

~/tensorflow/lib/python3.6/site-packages/keract/keract.py in eval_fn(k_inputs)
     89                 print(k_inputs)
     90                 print(nodes_to_evaluate)
---> 91                 return K.function(k_inputs, nodes_to_evaluate)(x)
     92             return K.function(k_inputs, nodes_to_evaluate)((x, y))  # although works.
     93         except ValueError as e:

~/tensorflow/lib/python3.6/site-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   3790         value = math_ops.cast(value, tensor.dtype)
   3791       converted_inputs.append(value)
-> 3792     outputs = self._graph_fn(*converted_inputs)
   3793
   3794     # EagerTensor.numpy() will often make a copy to ensure memory safety.

~/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
   1603       TypeError: For invalid positional/keyword argument combinations.
   1604     """
-> 1605     return self._call_impl(args, kwargs)
   1606
   1607   def _call_impl(self, args, kwargs, cancellation_manager=None):

~/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _call_impl(self, args, kwargs, cancellation_manager)
   1643       raise TypeError("Keyword arguments {} unknown. Expected {}.".format(
   1644           list(kwargs.keys()), list(self._arg_keywords)))
-> 1645     return self._call_flat(args, self.captured_inputs, cancellation_manager)
   1646
   1647   def _filtered_call(self, args, kwargs):

~/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
   1744       # No tape is watching; skip to running the function.
   1745       return self._build_call_outputs(self._inference_function.call(
-> 1746           ctx, args, cancellation_manager=cancellation_manager))
   1747     forward_backward = self._select_forward_and_backward_functions(
   1748         args,

~/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
    596               inputs=args,
    597               attrs=attrs,
--> 598               ctx=ctx)
    599         else:
    600           outputs = execute.execute_with_cancellation(

~/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     58     ctx.ensure_initialized()
     59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60                                         inputs, attrs, num_outputs)
     61   except core._NotOkStatusException as e:
     62     if name is not None:

InvalidArgumentError:  You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,256,256,3]
	 [[node input_1 (defined at /home/fernanda/tensorflow/lib/python3.6/site-packages/keract/keract.py:91) ]] [Op:__inference_keras_scratch_graph_29806]

Function call stack:
keras_scratch_graph

@nandaw can you paste a full snippet that I can run?

Sure, @philipperemy
import warnings
import numpy as np

warnings.filterwarnings("ignore")
np.random.seed(40)

import tensorflow as tf
from tensorflow.keras.applications import InceptionV3
from tensorflow.keras.layers import Dense, Dropout, GlobalAveragePooling2D, TimeDistributed, LSTM, Bidirectional, Lambda
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras import Input
import tensorflow.keras.backend as K

import itertools
import keract

import seaborn as sb

import sys
import os

import pandas as pd
import argparse

import cv2

os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ''

IMAGE_SIZE = 300
N_SLICES = 30

def build_model():

    base_model = InceptionV3(weights='imagenet',
                                   include_top=False,
                                   input_shape=(IMAGE_SIZE, IMAGE_SIZE, 3))
    last = base_model.output

    x = GlobalAveragePooling2D()(last)

    model1 = Model(base_model.input, x)

    input_tensor = Input(shape=(N_SLICES, IMAGE_SIZE, IMAGE_SIZE, 3))
    timeDistributed_layer = TimeDistributed(model1)(input_tensor)

    #the ouput of the model is [1, 1, 2048], need to squeeze
    # som = Lambda(lambda x: K.squeeze(K.squeeze(x,2),2))(timeDistributed_layer)

    bnn = Bidirectional(LSTM(1000))(timeDistributed_layer)
    bnn = Dropout(0.5)(bnn)

    bnn = Dense(512, activation='relu')(bnn)
    bnn = Dropout(0.3)(bnn)

    preds = Dense(3, activation='softmax', name='out')(bnn)

    model = Model(input_tensor, preds)

    return model

def compile_model(model, learning_rate=1e-5):
    """

    :param model: builded model
    :return: compiled model
    """

    optimizer = Adam(lr=learning_rate)
    model.compile(loss='binary_crossentropy',
                  optimizer=optimizer)
    return model

model = build_model()
model = compile_model(model, learning_rate=1e-4)

inputs = [np.random.uniform(size=(N_SLICES,IMAGE_SIZE, IMAGE_SIZE, 3))]

time_d = keract.get_activations(model, inputs,nested=True)

I spent more than 30min on it and could not find an easy to do what you want to do.
There seems to be a bug when you use an input that is not the input of the pre-trained model.
Model(inputs=base_model.input, outputs=x)

This would work but defining input_tensor = Input(shape=(N_SLICES, IMAGE_SIZE, IMAGE_SIZE, 3), name='hello') seems not to work and I don't know why.
I'll reopen this issue but I don't know when I can find time to work on that.

But that solution should save you for now:
import warnings

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Lambda
from tensorflow.python.keras.applications.inception_v3 import InceptionV3

from utils import print_names_and_shapes

tf.compat.v1.disable_eager_execution()
warnings.filterwarnings("ignore")
np.random.seed(40)

from tensorflow.keras.layers import Dense, Dropout, GlobalAveragePooling2D, LSTM, Bidirectional
from tensorflow.keras.models import Model
import tensorflow.keras.backend as K
from tensorflow.keras.optimizers import Adam

import keract

import os

os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ''

IMAGE_SIZE = 300
N_SLICES = 30

def build_model():
    base_model = InceptionV3(include_top=False)
    x = base_model.output
    x = GlobalAveragePooling2D()(x)  # x.shape = (batch_size*num_slices, 2048).
    # un-squeeze the first dimension to batch_size, and num_slices.
    # input.shape = (batch_size*num_slices, 2048)
    # output.shape = (batch_size, num_slices, 2048)
    l3 = Lambda(lambda z: K.reshape(z, (-1, N_SLICES, x.shape[-1])))(x)

    # input.shape = (batch_size, time_steps, input_dim=2048).
    bnn = Bidirectional(LSTM(1000))(l3)
    bnn = Dropout(0.5)(bnn)
    bnn = Dense(512, activation='relu')(bnn)
    bnn = Dropout(0.3)(bnn)
    preds = Dense(3, activation='softmax', name='out')(bnn)

    return Model(base_model.input, preds, name='m2')

def compile_model(model, learning_rate=1e-5):
    """

    :param model: builded model
    :return: compiled model
    """

    optimizer = Adam(lr=learning_rate)
    model.compile(loss='binary_crossentropy',
                  optimizer=optimizer)
    return model

model = build_model()
model = compile_model(model, learning_rate=1e-4)

# The first dimension contains actually two dimensions:
# - the slices dimension
# - the batch dimension
batch_size = 2
inputs = np.random.uniform(size=(N_SLICES * batch_size, IMAGE_SIZE, IMAGE_SIZE, 3))
print(model.predict_on_batch(inputs).shape)
activations = keract.get_activations(model, inputs, nested=True)
print_names_and_shapes(activations)

Output
[...]
concatenate_1 (60, 8, 8, 768)
activation_93 (60, 8, 8, 192)
mixed10 (60, 8, 8, 2048)
global_average_pooling2d (60, 2048)
lambda (2, 30, 2048)
bidirectional (2, 2000)
bidirectional/forward_lstm (2, 1000)
bidirectional/backward_lstm (2, 1000)
dropout (2, 2000)
dense (2, 512)
dropout_1 (2, 512)
out (2, 3)

I realized that doing
strongest = np.argmax(activations,axis=3)

for finding the strongest activation was incorrect.
Then i did:
# Return the index for the strong activation for a 4d array (1,channel_width,channel_length, n_features)
# Example below works for conv5_block3_out (ResNet50), from which output originally is (1,7,7,2048).
def find_strongest_activation(activations):
  outputs = np.squeeze(activations,axis=0) # unbatch array into a 3d one (7,7,2048).
  max = 0
  index = 0
  outputs = np.sum(outputs,axis = 0) # Sum rows and columns resulting into a 2d array (7,2048).
  for i in range(len(outputs[0][:])): # Then find max.
    sum = 0
    for row in range(7):
      sum = sum + outputs[row][i]
    if sum > max:
      max = sum
      index = i
  return index

And by plotting heatmaps i was able to see more intuitively things with respect the obtained features.

But that solution should save you for now. The trick is to use the default Input tensor provided by the pre-trained model and push extra information (push num_slices into the first dimension), calling the pre-trained model and un-squeeze the first dimension to recover batch_size and num_slices.

Only today I had the opportunity to implement and test your change. It worked like a charm! Huge thanks!!

@nandaw happy I could help!


Integration: DeepSource

ref: #212
Using DeepSource to continuously analyze your repository:

Install DeepSource on your repository here.
create a.deepsource.toml configuration which you can use to configure your analysis settings.
Activate analysis here.

This PR includes the .deepsource.toml file so you need not create the configuration file but if you want you can modify it any time either manually or update it from the dashboard as per your requirement.

@f213 :wave:
Just want to follow up on this.
Also, I work at DeepSource. So if you have any suggestions/advice regarding DeepSource you can let me know.


Add clang format

Per https://github.com/mapbox/hpp-skel/issues/31
Next Actions

[x] Add .clang-format
[x] Add ./scripts/format.sh
[x] Run ./scripts/format.sh against hpp-skel code
[x] Add clang-format Travis build
[x] Add CONTRIBUTING.md
[x] Update .gitignore per installing clang-format
[x] Update CHANGELOG.md
[x] Code Review
[ ] Merge after Travis is 🍏

cc @springmeyer @mapsam

Thanks so much for the thorough CR @springmeyer 👍  Implemented and waiting on Travis


some modifications about SmartContractZero and Chain initialization

"var smartContract = await runner.RunAsync(reg)"  the way I implement is directly create SmartContract using SmartContractRegistration, which needs the contract class written by user should implement ISmartContract just like the Contract Demo in AElf.Kernel.Test.

another point to be confirmed is that in my design for now, when a new contract deployed, a new account address will be generated which is associated with this contract. BTW, SmartContractRegistration is created in kernel, and hash of that is calculated with contract name and account who deploy it. And the hash value is also the address of new account.  The hash calculating method can be changed later.


Allow to override keybindings

Hey,
I’m a using a non QWERTY keyboard layout (bépo) and I need to redefine a bunch of keys (i j k l are redefined and I map them to c t s r, which means that those keys are important to me), so I need to remap things such as stage / rebase.
Is it possible? Thanks in advance.

This is not possible yet, but I will add this later today.

I added functionality for overriding mappings. Please look at the README for further information.
I will not close this issue until all popups support this.
Missing popups:

[ ] Help (Should be generated from status mappings I guess?)
[ ] Push
[ ] Pull
[ ] Branch
[ ] Commit
[ ] Log
[ ] Stash

Thanks a lot for the rapid feedback! :bow:

After thinking about it some more, I'm not too sure whether supporting remappings of popup windows is really needed.
@RianFuro any thoughts?

@TimUntersberger It's tricky, right? Magit has this "convention over configuration" mentality, which kind of clashes how stuff is usually done in vim-space.
I think you're right; we don't actually need remappable controls in popups as far as keyboard layouts are concerned, since all that's allowed in there is pressing the described shortcuts.
Beyond needing though, this becomes a discussion of how much we want to move away from that convention over configuration mindset. I don't think there's a clear answer to that; I'd likely not implement this myself since I see no need, but if somebody would feel strong enough about it to contribute the functionality, I wouldn't be entirely against merging it.

The idea is that I’m not using a qwerty keyboard, I’m using a bépo layout, and my j and k keys are t and s. So if I want to move the cursor up, I press s.
I would definitely like a way to override pop-ups keybindings as it will be a nightmare otherwise. The only other solution I have otherwise is to fork and modify the Lua code, which is clearly not ideal.

@phaazon you're not really moving the cursor (or doing anything but using the presented keybindings for that matter) in the popups.
Maybe we need a better naming scheme here :sweat_smile: when we say popup we mean the prompts that just show you the available options and keybindings, not the status or log buffer and such.
I'm just using a regular qwerty layout though, so maybe I'm overlooking something here. Can you elaborate what problems you would have in the prompt-style popups? I realize that cursor movement and other default bindings are different for you, but I don't think any of those apply in those buffers (or make no sense to use anyway)

@phaazon the way I think about the popups (or "transient commands" in magit) is a visual representation of key chords.
For example:
You could add a mapping for pushing to upstream like Pu which does exactly the same (Actually in magit the popups only appear after some delay after pressing P).
Now one advantage of popups is that you don't have to remember ANY keybindings except the popup keybindings, because you always have a list of available commands. Another one is that these make it possible to realize something like the options and switches you can see in some popups (like the log popup), which would be impossible to make usable with normal keybindings.
And don't get me wrong if there really is a need for being able to customize them, then we will implement it. Implementing this is really easy, because you just have to do the same thing I did for the status buffer except for the popups themselves (which is really tedious to maintain/create imo)

Navigating hunks / files does move the cursor, though.

@phaazon That's where the difference is. We're completely on board with allowing mapping customization for the status- and log-buffer (and every subsequent navigatable buffer). But as for the "shortcut" buffers (like when you press c for the commit popup or Z for the stash popup from the status buffer), there's really no need to customize anything, as they just offer you some mnemonic bindings and a way to set options and don't allow (or at least meaningfully react) to any other input.

Ah for the transient popups, I totally agree, it even makes more sense to keep semantic keybindings — i.e. ce for commit extend, etc.

@phaazon is this fixed for you?


A picture is worth 1,000 words

Add the gif to the readme.

Here is what it looks like in my branch now: https://github.com/breck7/virtual-unfolding

thanks for the idea!  I decided to go with a jpg still since the gif may be slow to load for some:
https://github.com/UnlockingHistory/virtual-unfolding

Good call! The gif is nice but the jpg conveys the same idea quicker and with less fancy technology :)


Release 0.9.2

Backport tasks for 0.9.2 release:

[x] - https://github.com/api3dao/airnode/issues/1583
[x] - https://github.com/api3dao/airnode/issues/1565

The release is blocked until all of the issues here are completed (cc: @aTeoke )

Created snapshot packages and Docker images for v0.9.2 and testing issue for them https://github.com/api3dao/airnode/issues/1588.
Once tested, I'll do a proper release.

The release is done. Moving to Testing.
@martinkolenic
All NPM packages and Docker images are available under the 0.9.2 tag (or latest at the moment).

Hi, @amarthadan, this has been tested. The test reports are below; unfortunately I automatically cleared the terminal screen when I was done before being able to stop myself so I have no screenshots. Please let me know if you wish me to rerun the tests with the screenshots. Airnode deployer screen did show v0.9.2.
1: Deploy and remove Airnode (Docker, npm) - PASS
Steps: I deployed (using the Docker image) and removed (using the npm package) Airnode with the coingecko-signed-data example. I then ran a POST request to the gateway to check that functionality.
Expected result: Successful deployment and meaningful POST request response
Actual result: Matches the expected result
2: Test admin (Docker, npm) - PASS
Steps: I tested mnemonic generation in both the npm package and the Docker image.
Expected result: Successful command execution
Actual result: Matches the expected result
3: Test validator (npm) - PASS
Steps: Run it against a valid and invalid config file
Expected result: Confirm the (in)validity of the config files
Actual result: Matches expected result


No reference/glossary/control-plane.md in Korean

This issue is to encourage a new contributor for Korean localization.
환영합니다 :)
This is a...

[x] Feature Request
[ ] Bug Report

Problem:
No Korean translation for
https://kubernetes.io/ko/docs/reference/glossary/?fundamental=true#term-control-plane
Proposed Solution:
Translate the page in Korean.
/website/content/en/docs/reference/glossary/control-plane.md

checkout to the latest Korean working branch  (currently, it is dev-1.15-ko.1, but you need to check the latest one), and make your working branch.

Copy control-plane.md in /website/content/en/docs/reference/glossary/
to /website/content/ko/docs/reference/glossary/

Translate English contents in the control-plane.md file into Korean

title: Control Plane (to Korean)
id: control-plane
date: 2019-05-12
full_link:
short_description: >
The container orchestration layer that exposes the API and interfaces to define, deploy, and manage the lifecycle of containers. (to Korean)
aka:
tags:

fundamental

The container orchestration layer that exposes the API and interfaces to define, deploy, and manage the lifecycle of containers. (to Korean)

Submit a PR for the Korean working branch
(a merged PR which you can refer: https://github.com/kubernetes/website/pull/15278)

Please leave a comment if you wish to provide a contribution to this issue.
A Korean localization team member hopes to assign this issue to a new contributor.
(Please don't start the job in advance)
You can get additional information on Korean localization from the following links.
(1) 문서 기여 방법: https://github.com/kubernetes/website/blob/master/README-ko.md
(2) 문서 한글화 가이드 동영상: https://www.youtube.com/watch?v=OTl8HBjxIhc
(3) 한글화 스타일 가이드: https://kubernetes.io/ko/docs/contribute/localization_ko/
(4) 한글화 팀 Slack 채널: Kubernetes Slack 가입 후, #kubernetes-docs-ko 채널 참여하여 문의
Page to Update:
https://kubernetes.io/ko/docs/reference/glossary/?fundamental=true#term-control-plane

/language ko
/good-first-issue
/help

This issue has been taken by @devholic
Thank you.


fix typo on step 2 of Initiating Failover

Why:
Found a small typo on documentation

What's being changed (if available, include any code snippets, screenshots, or gifs):
literally one letter :) from node to mode

Check off the following:

[x] I have reviewed my changes in staging (look for the "Automatically generated comment" and click the links in the "Preview" column to view your latest changes).
[x] For content changes, I have completed the self-review checklist.

Writer impact (This section is for GitHub staff members only):

[ ] This pull request impacts the contribution experience

[ ] I have added the 'writer impact' label
[ ] I have added a description and/or a video demo of the changes below (e.g. a "before and after video")

@amenocal Thanks so much for opening a PR! I'll get this triaged for review :zap:


chore: bump version to 1.2.0

Issue #, if available:
Description of changes:

Checklist

[ ] Meet tenets criteria
[ ] Update tests
[ ] Update docs
[ ] PR title follows conventional commit semantics

Breaking change checklist

RFC issue #:

[ ] Migration process documented
[ ] Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #119 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #119   +/-   ##
========================================
  Coverage    98.71%   98.71%
========================================
  Files           17       17
  Lines          546      546
  Branches        48       48
========================================
  Hits           539      539
  Misses           7        7

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dbfbf55...9e63042. Read the comment docs.


fix(wallet): show invoice final amount in activity list

Description:
Show invoice final amount in activity list
Motivation and Context:
fix #2047
How Has This Been Tested?
Follow steps in #2047 (see thread comments)
Types of changes:
Bug fix
Checklist:

[x] My code follows the code style of this project.
[x] I have reviewed and updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes where needed.
[x] All new and existing tests passed.
[x] My commits have been squashed into a concise set of changes.

Coverage remained the same at 19.425% when pulling abb587c9bb759e56a03251654bf0f08fe234b2bf on mrfelton:fix/2047-invoice-amounts into 020e63999bdd5d542bcfa8115e9aaad253f6334b on LN-Zap:master.

Coverage remained the same at 19.425% when pulling abb587c9bb759e56a03251654bf0f08fe234b2bf on mrfelton:fix/2047-invoice-amounts into 020e63999bdd5d542bcfa8115e9aaad253f6334b on LN-Zap:master.


[DKIM] Use of relaxed/simple cannonicalization can lead to invalid DKIM signatures for long header lines

This issue comes from mail-in-a-box upstream, see https://github.com/mail-in-a-box/mailinabox/issues/2239.
Copy-Paste summary of the issue:
Currently, mail in a box configures opendkim to use relaxed/simple canonicalization (ll34 https://github.com/mail-in-a-box/mailinabox/blob/main/setup/dkim.sh ). This can lead to verification issues with long To: headers; Specifically, whitespaces/\r\n/\n get injected, which let verification fail, see https://www.rfc-editor.org/rfc/rfc6376#section-3.4 and https://www.rfc-editor.org/rfc/rfc6376#section-3.5.
(I am currently a bit unsure why this is an issue for relaxed/simple being used; Technically this should be a simple/(relaxed|simple) issue).
This does only cause issues for very long To: headers.
Reproducing the issue
To test this:

start a test for a system using mail-in-a-box here: https://www.email-security-scans.org/ (select "store emails for me")
Wait for measurement mail to arrive and reply
See that DKIM verification failed
Download all data and extract it.
Run manual verification with dkimpy: sent-messages % cat 1.mbox | dkimverify -> signature verification failed
Edit the message and fix the broken headers (most likely To: having newlines added)
Rerun verification: sent-messages % cat 1.mbox | dkimverify -> signature ok

This issue also reproduces when sending to, e.g., google. To test that, start a test (or create a long to yourself), but this time before sending the test message, add a gmail address to the To: as well. You will see that the mail is not validated by Gmail anymore.
Suggested solution
Switch to relaxed/relaxed canonicalization for DKIM signing.

This is how a normal email looks...

This is the error as an example when sending to a lots of recipients

NOTE: I was testing my mail server with a tool ichdasich helped develop, and found this error... wich ichdasich graciously filed as a bug far better than i could.

According to RFC6376 Section 3.4
It should be better to run relaxed/relaxed, which allows some minor modification the mail-header.


Action fails silently

Thanks for putting this action together! It would be nice to have it fail when an error is returned from the curl command. A few examples here:
{"error_summary": "missing_scope/.", "error": {".tag": "missing_scope", "required_scope": "files.content.write"}}
{"error_summary": "path/conflict/folder/..", "error": {".tag": "path", "reason": {".tag": "conflict", "conflict": {".tag": "folder"}}, "upload_session_id": "pid_upload_session:xxx-uuu"}}

It should also check the output of /oauth2/token:
{"error": "invalid_grant", "error_description": "refresh token is malformed"}
{"error": "invalid_client: Invalid client_id or client_secret"}

vs
{"access_token": "sl....", "token_type": "bearer", "expires_in": 14400}

And the requests I'm seeing currently don't reply with JSON for most errors, e.g.:
Error in call to API function "files/upload": Must provide HTTP header "Authorization" or URL parameter "authorization".
Error in call to API function "files/upload": Invalid authorization value in HTTP header "Authorization": "...".  Expecting "Bearer <oauth2-access-token>
Error in call to API function "files/upload": HTTP header "Dropbox-API-Arg": path: The root folder is unsupported.


Required variables

Hello,
Is it possible to check if some variables were set on the pipeline using a YAML file?
Someone else had opened a request on VSTS forum: https://developercommunity.visualstudio.com/idea/365977/required-variables-in-build-definitions.html
But I need to check some variables now, and I can't figure out how can I do this using my YAML file, used for a VSTS extension.

@raphaelhagi -- Raphael, thank you for your question.  The link you mention has recently been updated with a potential answer.  You may also find answers here:

Azure DevOps Support Bot
Azure DevOps on Stack Overflow

Are you looking at a specific docs.microsoft.com article?

Yes! We achieve this using a Powershell script to check variables values as mentioned on the Developer Community thread. Thanks, @WilliamAntonRohm .


fixed isLoadFresh accessed non existant loads variable

In certain cases (plugins) isLoadFresh tries to access the loads variable, which simply isn't present in the given context.
This PR fixes this issue.

Thanks, was just working on this!


have the Type on CompositeAttributeConverter (feature request)

Hi,
I have a use case where I use CompositeAttributeConverter<T> as generic.
    @Convert(value = EmbeddedObject.class, graphPropertyType = Address.class)
    private Address address = new Address();

So, EmbeddedObject<T> is a reusable class, instead of declaring per class (Address, Car and so).
public class EmbeddedObject<T> implements CompositeAttributeConverter<T>{...}
Some time in runtime you need the class type on the method:
public T toEntityAttribute(Map<String, ?> value)
I would be much better if graphPropertyType  was injected as the following:
public T toEntityAttribute(Map<String, ?> value, Class<T> graphPropertyType  )
So I would be able to run graphPropertyType  .newInstance(), of getting other useful things from type.
I think this approach is much more flexible and let you be more "generic" when coding.
WDYT?
Thanks

Thanks for your patience, @avifatal and for the report.
I see your use case, but for that, I would need to introduce a corresponding entityAttributeType. graphPropertyType is for the other way ("which supported datatype should be stored on the node / relationship inside the database").
I don't want to reuse it for the purpose above and I am unsure about the added, additional complexity.


Update Follow Up intent

Environment details:
OS type and version: Ubuntu 18.04.4 LTS
Python version and virtual environment information: Python 3.6.9
dialogflow version: Latest
I'm trying to add training phrases to a Follow Up intent. Here is my code:
def list_intents(project_id, client, parent):
    intents = client.list_intents(parent)
    dict_intents = {}
    for intent in intents:
        dict_intents[intent.name] = intent.display_name
    return dict_intents

def update_intent(project_id, intent_name, training_phrases_parts):
    client = dialogflow.IntentsClient()
    parent = client.project_agent_path(project_id)
    intent_path = ''
    dict_intents = list_intents(project_id, client, parent)
    for key, value in dict_intents.items():
        if value == intent_name:
            intent_path = key
    intent = client.get_intent(
        intent_path, intent_view=dialogflow.enums.IntentView.INTENT_VIEW_FULL)
    training_phrases = []
    for training_phrases_part in training_phrases_parts:
        part = dialogflow.types.Intent.TrainingPhrase.Part(
            text=training_phrases_part)
        training_phrase = dialogflow.types.Intent.TrainingPhrase(parts=[part])
        training_phrases.append(training_phrase)
    intent.training_phrases.extend(training_phrases)
    response = client.update_intent(intent)
    return 'Updated'

But I get this error:
InvalidArgument: 400 com.google.apps.framework.request.BadRequestException: Read-only field 'root_followup_intent_name' cannot be set.
This code works fine with regular intents but when I try to update a Follow Up Intent it fails...
Any idea what I should do? Any suggestion?

@Juliopdata root_followup_intent_name the name attribute of the intent is a UUID defined by Dialogflow and cannot be changed.
This stackoverflow answer might solve your problem.
Please take a look, if this helps you to resolve this issue
https://stackoverflow.com/questions/52647517/dialogflow-invalid-argument-error-on-followup-intent-update

Hi Mike! Thanks for your response.
I have read that stackoverflow answer but I not sure if it is the same issue because I don't want to change the intent UUID, and also I don't want to update a training phrase. I want to add new training phrases.
It's strange because I can add training phrases to a NON Folllow-up intent but I can't do it to a Follow-ip intent.
Sorry if I missing something these are my first steps with Dialogflow and thanks again for your help.

Hi again! Finally I got it.
You have to add an update_mask:
intent.training_phrases.extend(training_phrases)
    update_mask = field_mask_pb2.FieldMask(paths=[intent.display_name, 'training_phrases'])

Whole script:
def list_intents(project_id, client, parent):
    intents = client.list_intents(parent)

    dict_intents = {}

    for intent in intents:
        dict_intents[intent.name] = intent.display_name

    return dict_intents

def update_intent(project_id, intent_name, training_phrases_parts):
    import dialogflow
    client = dialogflow.IntentsClient()
    parent = client.project_agent_path(project_id)
    intent_path = ''
    dict_intents = list_intents(project_id, client, parent)
    for key, value in dict_intents.items():
        if value == intent_name:
            intent_path = key
    intent = client.get_intent(
        intent_path, intent_view=dialogflow.enums.IntentView.INTENT_VIEW_FULL)
    training_phrases = []
    for training_phrases_part in training_phrases_parts:
        part = dialogflow.types.Intent.TrainingPhrase.Part(
            text=training_phrases_part)
        training_phrase = dialogflow.types.Intent.TrainingPhrase(parts=[part])
        training_phrases.append(training_phrase)
    intent.training_phrases.extend(training_phrases)
    update_mask = field_mask_pb2.FieldMask(paths=[intent.display_name, 'training_phrases'])
    response = client.update_intent(intent, update_mask=update_mask)
    return 'Updated'

Thanks again and I hope it will be useful to someone

@Juliopdata Glad you figured out! Thanks for your contribution.
I make a note of this and eventually add to docs somewhere.


Tests for accounts

adds missing file for updating an account
adds tests for creating, updating, and deleting accounts

CF #24 #30


Fix js error

Fix js error when there is no structuredText of startLine or structuredText of startLine has not text attribute.
Also fixes this issueon karma-coverage: https://github.com/karma-runner/karma-coverage/issues/167

Coverage remained the same at 97.523% when pulling aeed31615dc6051cd26c92ba4891694e6b84e275 on cenkingunlugu:master into bc84c315271a5dd4d39bcefc5925cfb61a3d174a on gotwarlost:master.

Also solves: https://github.com/gotwarlost/istanbul/issues/817

I have a PR for only part of these changes. These should be linked.

Does anyone figured why this happening? In my case, doing migration js>ts(x) and just started to seeing this error. Assuming some upstream is screwing the sourcemaps (webpack + babel (react) + typescript + karma)?

@FezVrasta yes this is not a fix, it's just a "silent" fail. I think there should be an error thrown instead that just looks better than "TypeError", which would tell you that the coverage report cannot be mapped to the source file.
In my case, this happens because I am using TypeScript and so the coverage does not really cover the source file. I'm trying to switch and use karma-typescript with coverage to make it work but been quite unsuccessful so far

+1 👍


How to disable ListItem (BulletList, OrderedList)

Markwon version:  4.3.0

I would like to know to disable ListItem.
I disabled the list items by referring to the tip area like below
TIP
MarkwonVisitor also allows overriding already registered nodes.
Thank you in advance.

i found solution like below
override fun configureParser(builder: Parser.Builder) {
    builder.enabledBlockTypes(Collections.emptySet())
}


fix: icx-proxy should only consult query string and referrer when run locally

The hardest part of this PR will be naming the new command-line argument...

Thanks!

icx-proxy moved to https://github.com/dfinity/icx-proxy


Add New Card Component

I would like to add this card component I made for people to use.

@all-contributors please add @thetechie7 for code


Make fetchValue, getValue and keyAlreadyExists faster and more swifty

TLDR:
Take advantage of Swift's Array.contains and Dictionary subscript which return optionals.
Use guard instead of large if blocks.
Replace inefficient for loops with Swift's array features.

The two functions getValue and fetchValue currently takes 3 steps:

Convert plist into Dictionary
Loop through Dictionary keys to find the inputted key
Get the value for the given key

At the moment step 2 is inefficient because it continues looping through the entire key array, even after we're found the key we're looking for.
So we could break out of this loop when we find the given key.
However, we can take it one step further and remove this step all together because dict[key] will return us the value if the key exists or it will return nil if the key doesn't exist. Combining steps 2 & 3 into one, more efficient, step.

The function keyAlreadyExists, again continues looping and taking up resources even after we've identified that the key does exist.
Rather than using a for loop to match the key String, we can use Swift's Array.contains function which will tell us if the key exists, or return nil if it does not :)

Thanks for the update


Premint: fix fork tests

Current dependencies on/for this PR:

main

PR #161

PR #164   👈
PR #163
PR #162

This comment was auto-generated by Graphite.


feat: Auto enable integrations=true

We want to make auto enabling integrations the new default.

<3 thanks folks

We really need to improve CI build times


[BUG] PURL_functions acceptance test asserts over log files with zoned timestamp

Describe the Bug
The snapshot differs based on the time zone of the computer the acceptance tests are running on.
Steps to Reproduce

cd acceptance
go test -test.run 'TestFeatures/PURL_functions

Expected Behavior
A description of what you expected to happen.
Actual Behavior
--- FAIL: TestFeatures/PURL_functions (6.75s)
    suite.go:527: 1 error occurred:
        	*
        - Snapshot - 0
        + Received + 1

        - time="${TIMESTAMP}" level=error msg="Parsing PURL \"this-is-not-a-valid-purl\" failed: purl scheme is not \"pkg\": \"\""
        + time="${TIMESTAMP}+01:00" level=error msg="Parsing PURL \"this-is-not-a-valid-purl\" failed: purl scheme is not \"pkg\": \"\""
          Error: success criteria not met
          ↵

        at ../../../../../../../../../../../../../tmp/tmp.GHLYxwb0uo/features/__snapshots__/validate_image.snap:3342

Possible Solution
Looks like custom formatter should be implemented.

NOTE: When this was previously encountered, a change was made to the regex. That might also be the answer here.
If we go with the approach of a custom formatter, let's be sure to clean that regex.


Use cache for improved speed

Either

activate caching within the action,
or use lean-action to take advantage of the fact that it already has caching built in.

this issue should be closed as completed?


feat: The support for Chinese doesn't seem very good yet. Here is a test document.

The support for Chinese doesn't seem very good yet. Here is a test document:
https://github.com/digoal/blog/blob/master/202406/20240620_01.md

Thanks. We'll add more tokenizer here


kubeconfig connect failed

Hi all,
our cluster ran for exactly one year and now I can no longer log in with the previously used kubeconfig

after searching around i run the following to figure out the expiration of my local kubeconfig cert
 grep 'client-certificate-data' ./kubeconfig | awk '{print $2}' | base64 -d | openssl x509 -text

ok that might explain the root cause.
Can anyone please help and tell me how to fix this issue
thanks in advance..

Hi, looks like you need to rotate the certificates, see https://docs.k3s.io/cli/certificate
I thought that they had changed the validity to 10 years but according to that page it's still only 1 year. Try the commands in that page and let me know how it goes.

Hallo vitobotta,
thanks for the quick response...
yes i was at that point already but skipped it because after updating my local kubeconfig with the one from the master1 node /etc/rancher/k3s/k3s.yaml i could at least connect again to the cluster. But the both worker nodes were not working properly anymore. Half of the pods were terminating.

So I connected per ssh to master node 1 and followed the k3s docu on cert rotation. I cannot say if this changed anything because the worker nodes were still not accessible using kubectl. So I tried to drain what also not worked without a lot of error messages and than simply just rebooted both worker nodes.
Now the cluster is running again but I am already dreading the next event in a year ;-).
Is there any configuration to let k3s use longer expiration for their certs? The docs saying the cert rotation will happen even on k3s service restart but what sense could that make if guys like me are enjoinging to run the cluster for more than a year.

What I am confused about is, did you ever upgrade k3s in that past year? If not that's the reason why you ended up with expired certificates. If you upgrade k3s every couple of months the certificate rotation is done automatically and you won't happen to be again in this kind of situation. I am converting this to a discussion since it's something related to k3s, not specific to my tool.


SHIFT key does not work correctly in TextField widget (Flutter 2.0.1 Desktop Windows - Beta Channel)

Hi,
When starting to write in TextField widget, the SHIFT key is clicked at the same time that a key is pressed to display the uppercase character, when the SHIFT key is released the following characters are still written in uppercase, when they should be written in lowercase.
If I try to write "España", when I click on the shift key to capitalize the character "E", then I release the shift key, but all the rest of the characters are also capitalized "ESPAÑA".
Therefore, flutter desktop windows cannot be used to develop desktop Windows applications for production.
Neither Flutter version of either channel solves this problem.
Regards,
Jose.

Duplicate of https://github.com/flutter/flutter/issues/75675

flutter desktop windows cannot be used to develop desktop Windows applications for production

It's a beta; it's not yet intended for production use.

But they are very severe problems that make it completely impossible to make applications a little useful at the moment. :(
I think that in a beta this kind of obvious problems should be solved quickly.
I have seen in many of your current information that you put examples of multiplatform applications, but with these types of problems they can give the product very bad publicity ...
Thanks Stuart.
Regards,
Jose


How do I trap errors from calling a C++ function from Lua but with wrong parameter list

Similar to #910
I have a simple C++ function with say 3 parameters. I bind that function so that a Lua script can call it. The Lua script calls the function with an incorrect number of parameters. The function on the C++ side is never called and there are no exceptions thrown. How can I trap this situation so that I can report an error and the script writer can take proper action?
I've had a thorough look at the following docs but I'm still not understanding how to handle this situation:
https://sol2.readthedocs.io/en/latest/errors.html
https://sol2.readthedocs.io/en/latest/functions.html
https://sol2.readthedocs.io/en/latest/safety.html#config
https://github.com/ThePhD/sol2/blob/develop/examples/source/tutorials/quick_n_dirty/running_lua_code_safely.cpp#L36
https://sol2.readthedocs.io/en/latest/api/protected_function.html
Any help would be greatly appreciated.

Here is a reference to the docs on protected functions and error handlers for those that may stumble upon this. https://sol2.readthedocs.io/en/latest/api/protected_function.html


existing template files have ESLint errors

app.js:
ESLint  Error  object-curly-spacing A space is required after '{'.at line 2 col 8
Not a big deal, but what should be the standard; ESLint rules, or the files?

@egoist
Hope this can be fixed. Mostly the breakage is the space before function definition rule and the use of global variable location in pwa.js.


More pushing sql

add support for pushing case statements (plus test it)
remove support for pushing groupby (only whole-relation aggregates) because Myria is MUCH faster.

Coverage decreased (-0.02%) to 89.45% when pulling d40bd0064af7241be1db8ccb167d6cd5ce87296a on more-pushing-sql into bc2fe718c8c7cd78e27498c137b40f115c483077 on master.

Coverage increased (+0.02%) to 89.5% when pulling 3c29040f5992614c8f06ac859bd8a11e4aa7586a on more-pushing-sql into bc2fe718c8c7cd78e27498c137b40f115c483077 on master.


Feature Request - integration of LazyLibrarian python ebook grabber

DobyTang is actively maintaining LazyLibrarian and it is smokin hot because Integration with the default NZB and torrent grabbers are supported so i would like to have it included in HTPC as a standard app. Its python based and works smashing out of the box including integration into my reversed proxy using SSL. Back to you now, Hellowlol, you rock!

There is a pr for this. See https://github.com/Hellowlol/HTPC-Manager/pull/443

Holy smokes, #443 is closed, this makes me cry like a baby :'( This is precious code wasted, hurts my gut feeling seeing that happen. Somebody do something about it ....
Maybe that somebody should be me? Let me give it a try :-|

Let me know if you need any help.


Fix QML-related issues

Fixes some issues that I encountered while trying to implement some client code in QML.

 Mostly some trivial things, but there were some more serious issues:

Nested classes cannot be QObject or even Q_GADGET, so I had to workaround this for Key::Identifier. It's ugly but I didn't have a better idea.
QVector<> only becomes a JavaScript Array implicitly with some pre-defined template arguments, but not with custom types. So I had to add an ugly hack which involves QVariant
Also custom Q_GADGET types cannot be returned, those also had to be wrapped in QVariant

So, just the usual QML bulls*** that you have to deal with in every project.
On the upside, the signing flow now works with the QML API, which was previously impossible to use. I also added a new test which uses the QML-based test suite.

A couple of minor style comments, aside from that LGTM.  I'll ask Bea to review also.

@chriadam @blammit Addressed your concerns and rebased on top of current master. Please review and merge if you think it's ok now. :)

LGTM, thanks.  Feel free to merge if all unit tests pass on device etc.


File system inside container doesn't shrink

Required information
  driver: lxc
  driverversion: 2.0.1
  kernel: Linux
  kernelarchitecture: x86_64
  kernelversion: 4.4.0-22-generic
  server: lxd
  serverpid: 15964
  serverversion: 2.0.2
  storage: zfs
  storageversion: "5"

Issue description
zfs file system inside container provisioned by lxc from pool on host doesn't shrink. I tried setting a limit on the size of the root disk via lxc and a quota on the file system via zfs, and got:
error: Failed to set ZFS config: cannot set property for 'lxd/containers/postgres': size is less than current used or reserved space

Here is df -h from inside the container:
Filesystem               Size  Used Avail Use% Mounted on
lxd/containers/postgres  119G   47G   72G  40% /

Here is df -h from the host:
Filesystem                      Size  Used Avail Use% Mounted on
lxd/containers/postgres         119G   47G   72G  40% /var/lib/lxd/containers/postgres.zfs

Here is zfs list from the host:
NAME                                  USED  AVAIL  REFER  MOUNTPOINT
lxd                                   121G  71.9G    19K  /lxd
lxd/containers                        118G  71.9G    19K  /lxd/containers
lxd/containers/postgres              117G  71.9G  47.0G  /var/lib/lxd/containers/postgres.zfs

Is there another way to reduce the size of the container's file system?

Removing files is usually a good way to reduce disk usage :)

Your container contains 47G of stuff, setting a quota smaller than that isn't going to work until you remove some files.

I want to keep the 47G but reduce the used from zfs list from 117G down to the 47G - still likely to be something simple though :)

Can you paste "zfs list -t all"?

NAME                                                                                USED  AVAIL  REFER  MOUNTPOINT
lxd                                                                                121G  71.9G    19K  /lxd
lxd/containers                                                                                118G  71.9G    19K  /lxd/containers
lxd/containers@snap01                                                                                1K      -    19K  -
lxd/containers@snap02                                                                                1K      -    19K  -
lxd/containers@snap04                                                                                1K      -    19K  -
lxd/containers/nginx                                                                                398M  71.9G   993M  /var/lib/lxd/containers/nginx.zfs
lxd/containers/nginx@snapshot-nginx-backup                                                              322K      -   984M  -
lxd/containers/nginx@snapshot-nginx-backup-20160511                                                     328K      -   984M  -
lxd/containers/nginx@snapshot-nginx-backup-20160512                                                    3.73M      -   984M  -
lxd/containers/nginx@snapshot-migrated-live-careercalifornia-site                                      1.14M      -   985M  -
lxd/containers/nginx@snap01                                                                             267K      -   985M  -
lxd/containers/nginx@snap02                                                                             268K      -   985M  -
lxd/containers/nginx@snap04                                                                            64.1M      -   987M  -
lxd/containers/postgres                                                                                117G  71.9G  47.0G  /var/lib/lxd/containers/postgres.zfs
lxd/containers/postgres@snap04                                                                         3.35G      -  71.5G  -
lxd/containers/postgres@snapshot-backup-2016-05-27                                                     2.89G      -  71.0G  -
lxd/containers/studentfiles                                                                             172M  71.9G   904M  /var/lib/lxd/containers/studentfiles.zfs
lxd/containers/studentfiles@copy-0b2208e7-72b0-46f2-80cb-93f57e109e58                                  60.6M      -   901M  -
lxd/deleted                                                                                1.82G  71.9G    19K  /lxd/deleted
lxd/deleted@snap01                                                                                1K      -    19K  -
lxd/deleted@snap02                                                                                1K      -    19K  -
lxd/deleted@snap04                                                                                1K      -    19K  -
lxd/deleted/containers                                                                                1.19G  71.9G    19K  /lxd/deleted/containers
lxd/deleted/containers@snap01                                                                             1K      -    19K  -
lxd/deleted/containers@snap02                                                                             1K      -    19K  -
lxd/deleted/containers@snap04                                                                             1K      -    19K  -
lxd/deleted/containers/0ad619b2-9b29-45f6-a891-40a955d92c72                                            4.19M  71.9G   984M  none
lxd/deleted/containers/0ad619b2-9b29-45f6-a891-40a955d92c72@copy-efee7c47-6187-4db3-bccc-9ee4625f30c0    38K      -   984M  -
lxd/deleted/containers/0ad619b2-9b29-45f6-a891-40a955d92c72@snap01                                        1K      -   984M  -
lxd/deleted/containers/0ad619b2-9b29-45f6-a891-40a955d92c72@snap02                                        1K      -   984M  -
lxd/deleted/containers/0ad619b2-9b29-45f6-a891-40a955d92c72@snap04                                         0      -   984M  -
lxd/deleted/containers/3110522f-270b-44fa-8376-d3762aa3514c                                             226M  71.9G   902M  none
lxd/deleted/containers/3110522f-270b-44fa-8376-d3762aa3514c@snap01                                      338K      -   899M  -
lxd/deleted/containers/3110522f-270b-44fa-8376-d3762aa3514c@snap02                                      339K      -   899M  -
lxd/deleted/containers/3110522f-270b-44fa-8376-d3762aa3514c@snap04                                     70.2M      -   901M  -
lxd/deleted/containers/3110522f-270b-44fa-8376-d3762aa3514c@copy-58a4ef84-f215-4e3a-90a9-5f00f690329d   413K      -   902M  -
lxd/deleted/containers/b9a89886-4ae6-49ad-a2f6-f332babdd5e5                                             988M  71.9G   984M  none
lxd/deleted/containers/b9a89886-4ae6-49ad-a2f6-f332babdd5e5@copy-1880c049-0d3b-4ab8-8885-ed35211ca506  4.31M      -   987M  -
lxd/deleted/containers/b9a89886-4ae6-49ad-a2f6-f332babdd5e5@snap01                                        1K      -   984M  -
lxd/deleted/containers/b9a89886-4ae6-49ad-a2f6-f332babdd5e5@snap02                                        1K      -   984M  -
lxd/deleted/containers/b9a89886-4ae6-49ad-a2f6-f332babdd5e5@snap04                                         0      -   984M  -
lxd/deleted/containers/c2b6496b-2f83-4bd7-b303-b22cbbbebabc                                            56.5K  71.9G   902M  none
lxd/deleted/containers/c2b6496b-2f83-4bd7-b303-b22cbbbebabc@copy-2baf85f2-14d7-41cc-b932-5c1260bcd24e      0      -   902M  -
lxd/deleted/images                                                                                640M  71.9G    19K  /lxd/deleted/images
lxd/deleted/images@snap04                                                                                9K      -    19K  -
lxd/deleted/images/d23ee1f4fd284aeaba6adeb67cccf7b871e96178d637fec96320aab7cc9634b1                     640M  71.9G   640M  none
lxd/deleted/images/d23ee1f4fd284aeaba6adeb67cccf7b871e96178d637fec96320aab7cc9634b1@readonly              1K      -   640M  -
lxd/deleted/images/d23ee1f4fd284aeaba6adeb67cccf7b871e96178d637fec96320aab7cc9634b1@snap04                 0      -   640M  -
lxd/images                                                                                1.50G  71.9G    19K  /lxd/images
lxd/images@snap01                                                                                1K      -    19K  -
lxd/images@snap02                                                                                1K      -    19K  -
lxd/images@snap04                                                                                1K      -    19K  -
lxd/images/793122da8bfc432366179bf38bda488750e41e9d9499d85e3343f62b60dbf16c                             895M  71.9G   895M  /var/lib/lxd/images/793122da8bfc432366179bf38bda488750e41e9d9499d85e3343f62b60dbf16c.zfs
lxd/images/793122da8bfc432366179bf38bda488750e41e9d9499d85e3343f62b60dbf16c@readonly                      1K      -   895M  -
lxd/images/793122da8bfc432366179bf38bda488750e41e9d9499d85e3343f62b60dbf16c@snap01                        1K      -   895M  -
lxd/images/793122da8bfc432366179bf38bda488750e41e9d9499d85e3343f62b60dbf16c@snap02                        1K      -   895M  -
lxd/images/793122da8bfc432366179bf38bda488750e41e9d9499d85e3343f62b60dbf16c@snap04                         0      -   895M  -
lxd/images/fa117e8fba4533655e47d5429e3150f68fd0b03f3d6020d67dad857e7513e35e                             641M  71.9G   641M  /var/lib/lxd/images/fa117e8fba4533655e47d5429e3150f68fd0b03f3d6020d67dad857e7513e35e.zfs
lxd/images/fa117e8fba4533655e47d5429e3150f68fd0b03f3d6020d67dad857e7513e35e@readonly                       0      -   641M  -

zfs space reporting is pretty confusing, but the most likely suspects are those two snapshots.

that makes sense - i see they are referring to the 71.5G.


Remove from future import *

I'm not 100% sure editing the migrations totally make sense.  They get generated automatically by manage.py makemigrations.  But I guess maybe when Django 2 drops Python 2 support, that will change, and we'll only end up with a few leaking back in before that.
So, merged, thanks @rht!


fix: tentative to fix race condition

Experiment to try fixing race condition that seems to have been introduced by https://github.com/dagger/dagger/commit/82b67892fa47038e85f7bf226e25dfedce0aa320.
cf. corresponding trace: https://dagger.cloud/dagger/traces/c71cecd3d42758687c7437e4b7ec98d1?span=1302c9920a784c73
Not the best implementation but fixes the go races

SDK errors are unrelated, local runs do work on my side

Race got triggered again, possible that this is not the root cause

closing as this is probably unrelated