ISPN-15186 SIFS may not shut down properly with compactor still running

https://issues.redhat.com/browse/ISPN-15186
https://issues.redhat.com/browse/ISPN-15196
Also includes the issue where the compactor can attempt to compact an entry that does not have an index yet due to timing.

Also to note the StressTest has passed multiple times for me now without any issues. Note that tracing on or off can mess with timeout.

Yes, good catch. Will update on Monday.
On Fri, Sep 29, 2023, 3:52 PM José Augusto Bolina @.***>
wrote:

@.**** approved this pull request.
LGTM. Pretty nice.
In
core/src/test/java/org/infinispan/persistence/sifs/SoftIndexFileStoreTest.java
https://github.com/infinispan/infinispan/pull/11338#discussion_r1341816074
:

 fork(() -> compactor.performExpirationCompaction(expSub));

 // Compaction shouldn't complete as indexing wasn't allowed to proceed

 assertFalse(latch.await(100, TimeUnit.MILLISECONDS));

 // Finally let the indexing continue

 processors[0] = original;

 queue.forEach(original::onNext);

 // Now expiration compaction should finish as indexing will occur

 assertTrue(latch.await(10, TimeUnit.SECONDS));

 // The stop method blocks for NBSIS as we are faking the blocking manager

 Future<CompletionStage<Void>> stage = fork(() -> store.stop());

 eventually(stage::isDone);

This seems to assert only the Future and not the CompletionStage.
—
Reply to this email directly, view it on GitHub
https://github.com/infinispan/infinispan/pull/11338#pullrequestreview-1651515595,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE6H32CKIUXDTPQCHFILITX45GJ3ANCNFSM6AAAAAA5LVEMFI
.
You are receiving this because you authored the thread.Message ID:
@.***>

Seems CI crashed:
14:27:03  13:27:03.146 [ERROR] Errors:
14:27:03  13:27:03.146 [ERROR]   PessimisticTxPartitionAndMergeDuringPrepareTest.testDegradedPartition:29->BasePessimisticTxPartitionAndMergeTest.doTest:49->BaseTxPartitionAndMergeTest.createKeys:131 » IllegalState Could not find any segment owned by Cache 'pes-cache'@PessimisticTxPartitionAndMergeDuringPrepareTest-NodeC, [Cache 'pes-cache'@PessimisticTxPartitionAndMergeDuringPrepareTest-NodeB], primary segments: {33-44 48-59 153-174 185-194 218 234-235 242-244}, backup segments: {Cache 'pes-cache'@PessimisticTxPartitionAndMergeDuringPrepareTest-NodeB={0-11 22-23 74-78 95-105 112-113 127-130 136-146 206-217 249-250 253-255}}
14:27:03  13:27:03.146 [INFO]
...
14:27:03  13:27:03.157 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.1.2:test (default-test) on project infinispan-core:
14:27:03  13:27:03.157 [ERROR]
14:27:03  13:27:03.157 [ERROR] Please refer to /home/infinispan/workspace/Infinispan_PR-11338/core/target/surefire-reports for the individual test results.
14:27:03  13:27:03.157 [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
14:27:03  13:27:03.157 [ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?

Going to replay.

@wburns The SFSToSIFSMigratorTest are related?

@wburns did you have a look at the failures ?

It is related, underlying cause is:
09:38:36,958 ERROR (blocking-thread-SFSToSIFSMigratorTest-NodeA-p13-t2:[]) [o.i.p.s.Index] Error encountered with index, SIFS may not operate properly.
java.lang.AssertionError: null
	at org.infinispan.persistence.sifs.IndexNode.copyWith(IndexNode.java:780) ~[classes/:?]
	at org.infinispan.persistence.sifs.IndexNode.setPosition(IndexNode.java:418) ~[classes/:?]
	at org.infinispan.persistence.sifs.Index$Segment.accept(Index.java:563) ~[classes/:?]
	at org.infinispan.persistence.sifs.Index$Segment.accept(Index.java:456) ~[classes/:?]
	at io.reactivex.rxjava3.internal.subscribers.LambdaSubscriber.onNext(LambdaSubscriber.java:65) ~[rxjava-3.1.4.jar:?]
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:404) ~[rxjava-3.1.4.jar:?]
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178) ~[rxjava-3.1.4.jar:?]
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324) ~[rxjava-3.1.4.jar:?]
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289) ~[rxjava-3.1.4.jar:?]
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250) ~[rxjava-3.1.4.jar:?]
	at org.infinispan.util.concurrent.BlockingManagerImpl.lambda$asExecutor$9(BlockingManagerImpl.java:336) ~[classes/:?]
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) [jboss-threads-3.5.0.Final.jar:3.5.0.Final]
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) [jboss-threads-3.5.0.Final.jar:3.5.0.Final]
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) [jboss-threads-3.5.0.Final.jar:3.5.0.Final]
	at java.lang.Thread.run(Thread.java:833) [?:?]

So this is actually a bug in the SFSToSIFSStore migration class that is exposed in this PR.
https://github.com/infinispan/infinispan/blob/b28d3509a72f5bf62229034927e31b9bbc300c55/core/src/main/java/org/infinispan/configuration/parsing/SFSToSIFSStore.java#L67
This line is accidentally taking the segment from the MarshallableEntry instead of the key. The odd thing is I am not sure why the test passed before, it must have ignored the segment when doing compaction... I will look a bit closer.
09:40:54,686 TRACE (blocking-thread-SFSToSIFSMigratorTest-NodeA-p8-t4:[]) [o.i.p.s.Index] Indexing IndexRequest{key=key, serializedKey=ByteBufferImpl{length=13, offset=0, bytes=9801028A01070A05\J03\k\e\y (13 bytes)}, cacheSegment=213, file=0, offset=0, prevFile=-1, prevOffset=-1, size=55, type=UPDATE}
09:40:54,696 TRACE (blocking-thread-SFSToSIFSMigratorTest-NodeA-p8-t4:[]) [o.i.p.s.Index] Indexing IndexRequest{key=key, serializedKey=ByteBufferImpl{length=13, offset=0, bytes=9801028A01070A05\J03\k\e\y (13 bytes)}, cacheSegment=213, file=0, offset=55, prevFile=-1, prevOffset=-1, size=55, type=UPDATE}
09:40:58,136 TRACE (blocking-thread-SFSToSIFSMigratorTest-NodeA-p13-t2:[]) [o.i.p.s.Index] Indexing IndexRequest{key=key, serializedKey=ByteBufferImpl{length=13, offset=0, bytes=9801028A01070A05\J03\k\e\y (13 bytes)}, cacheSegment=99, file=0, offset=55, prevFile=0, prevOffset=0, size=-1, type=DROPPED}

As you can see from the TRACE the cacheSegment doesn't match, it is supposed to be 99

Test should be fixed with inclusion of fix for ISPN-15222

Thanks, @wburns!


Camera3D camera target Y value NaN

Camera3D camera target's Y component is NaN in the FPS camera mode. It causes calculations with it to be NaN.

Sorry I didn't respond to this issue sooner.
I need a example to test this issue. For example, how are you setting up the camera. Is it similar to core_3d_camera_first_person etc.


node.ts: implement Node.childElementCount

Implement Node.childElementCount.
The Document.childElementCount read-only property returns the number of child elements of the document.
https://developer.mozilla.org/en-US/docs/Web/API/Element/childElementCount
https://developer.mozilla.org/en-US/docs/Web/API/Document/childElementCount

Updated!

Sorry, I just noticed that childElementCount is not implemented for Node and is instead implemented on Element and Document... It'd be great if you could move it to element.ts and document.ts, otherwise I can merge it and do it later.

Moved it to both document.ts and element.ts. Removed it from node.ts. Updated unit tests.
Also, added .vscode to .gitignore so it didn't pick up the .vscode directory created by the deno vscode extension. Seemed like there was a precedent for this because of the .vim in there.


Attribute text on a html tag overrides inner content

Vue version
3.5.12
Link to minimal reproduction
https://stackblitz.com/edit/vitejs-vite-v477mx?file=src%2FApp.vue
Steps to reproduce
Add to html tag a attribute text
What is expected?
attribute text does not override content html tag a
What is actually happening?
attribute text overrides content html tag a
System Info
No response
Any additional comments?
No response

same issue on 3.4.7

Thanks for reporting this.
The issue occurs because Vue sets the text as a property on the  element, and while  doesn't have a text attribute, it does have a text property via HTMLAnchorElement(https://developer.mozilla.org/en-US/docs/Web/API/HTMLAnchorElement/text). This causes the property to override the inner content.
As a workaround, you can use ^text to prevent Vue from setting text as a prop. This will ensure that the text value is treated as an attribute instead.

Thanks for the lightning-fast response and fix! 🤝

Thank you for the clarification.
To explain my use case a bit more, I'm encountering the issue with overriding inner content when using v-bind in combination with our CMS component, which maps its own fields to an  HTML element (e.g., for a banner).
Here’s an example of what I'm working with:
const cmsBanner = {
  "id": "ui_banner_1",
  "text": "Some description",
  "target": "_blank",
  "href": "/some-href",
};

In this scenario, I’m not sure how the ^text solution would apply or help, since the text is part of the CMS data structure. Could you provide further clarification on how I might implement ^text in this case, or suggest an alternative approach to prevent the text property from overriding the inner content of the anchor tag?
Thank you in advance!

@jh-leong thanks a lot for the quick fix

https://github.com/vuejs/core/commit/8084156f4d0b572716a685a561d5087cddceab2c
In addition, it seems to be related to this commit. After the modification, mout children first and then update props, so it is overwritten.

In this scenario, I’m not sure how the ^text solution would apply or help, since the text is part of the CMS data structure. Could you provide further clarification on how I might implement ^text in this case, or suggest an alternative approach to prevent the text property from overriding the inner content of the anchor tag?

If the text property isn't needed, you can remove it before binding like this:
const cmsBanner = {
  id: 'ui_banner_1',
  text: 'Some description',
  target: '_blank',
  href: '/some-href',
};
const obj = { ...cmsBanner };
delete obj.text;

Playground

You can also do it this way to avoid the delete.
const { text, ...obj } = cmsBanner;

In this scenario, I’m not sure how the ^text solution would apply or help, since the text is part of the CMS data structure. Could you provide further clarification on how I might implement ^text in this case, or suggest an alternative approach to prevent the text property from overriding the inner content of the anchor tag?

If the text property isn't needed, you can remove it before binding like this:
const cmsBanner = {
  id: 'ui_banner_1',
  text: 'Some description',
  target: '_blank',
  href: '/some-href',
};
const obj = { ...cmsBanner };
delete obj.text;

Playground

You can also do it this way to avoid the delete.
const { text, ...obj } = cmsBanner;

Sure. I've done it this way. However, it's just a workaround and not a solution.

Sure. I've done it this way. However, it's just a workaround and not a solution.

I don't think it's a workaround because I don't think there's a bug here.   is the markup representation of HTMLAnchorElement, which has a text property.  Setting that changes the contents of the anchor.  Everything here seems to be working as documented.  Why do you expect it to work differently?

Sure. I've done it this way. However, it's just a workaround and not a solution.

I don't think it's a workaround because I don't think there's a bug here.  is the markup representation of HTMLAnchorElement, which has a text property. Setting that changes the contents of the anchor. Everything here seems to be working as documented. Why do you expect it to work differently?

Thank you for addressing this property. I had initially assumed it would behave slightly differently. For instance, if I wanted to set the text content of a link using JavaScript, I might use something like this:
document.querySelector('a').text = 'Some text content';

Here, text is a synonym for textContent, making it a straightforward way to update the text of a link.
However, in this particular case, text is not treated as a property in Vue but rather as an attribute. To verify this behavior, I tested adding the text attribute directly to an  tag in plain HTML (without Vue), and it worked as expected. You can find a demonstration of this behavior in the example below:
CodePen Example
This highlights a distinction between how Vue handles attributes and properties for DOM elements.

This is a difference between vue templates and HTML.   is not special.  <input type="checkbox" indeterminate> works in a vue template.  There's no way to achieve that at all in HTML without javascript.
I don't know where or if this distinction is mentioned in the docs, but this might be working as intended.


Update bb commit reference to renamed aztec-2.0

This branch is just here to back port the Noir backend as we update to the new barretenberg repository. The old repo was renamed and we need this new commit reference in order for the main Noir repo to pass its CI build.

Closing as we no longer reference the old backend


Show discount on product page

What is the motivation for adding / enhancing this feature?

I have lower price after adding a product to the shoping cart. Would it be possible to see discount on the product page before adding to shoping cart. There is some misinformation about actual price of the product.
What are the acceptance criteria

Product discount should be show on the product page

Can you complete this feature request by yourself?
NO
Additional information

@KarolaMikula I guess it depends. At least in M2 we have catalog price rules where discount are visible on catalog/product pages or cart rules where discount apply only in the cart. Someone need to check how it is done in VSF. (perhaps me tomorrow)

Take a look into #2584
I investigated payment system there. From the cart perspective, but I belive it will give you some insight about how the prices are calculated.

If there is a catalog pricing rule the discount is displayed; we can't access shopping cart rules (this particular case) to show the price after product has been added to cart
Closing for now


Activating Open Collective

Hi, I'm making updates for Open Collective. Either you or another core contributor signed this repository up for Open Collective. This pull request adds financial contributors from your Open Collective https://opencollective.com/fuse-studio ❤
What it does:

adds a badge to show the latest number of financial contributors
adds a banner displaying contributors to the project on GitHub
adds a banner displaying all individuals contributing financially on Open Collective
adds a section displaying all organizations contributing financially on Open Collective, with their logo and a link to their website

P.S: As with any pull request, feel free to comment or suggest changes.
Thank you for your great contribution to the Open Source community. You are awesome! 🙌
And welcome to the Open Collective community! 😊
Come chat with us in the #opensource channel on https://slack.opencollective.com - great place to ask questions and share best practices with other Open Source sustainers!

Thank you!
I've copied this information to some of our other repositories that are currently more active.

Superseded by https://github.com/fuse-open/fuse-studio/pull/105.


.draw(output = 'mpl"), plot_bloch_multivector etc. produce two plots even after removing %matplotlib inline

Informations

Qiskit version:
'qiskit': '0.29.0'
'qiskit-terra': '0.18.1',
'qiskit-aer': '0.8.2',
'qiskit-ignis': '0.6.0',
'qiskit-ibmq-provider': '0.16.0',
'qiskit-aqua': '0.9.4',
'qiskit-nature': '0.1.5',
qiskit-finance': '0.2.0',
'qiskit-optimization': '0.2.1',
'qiskit-machine-learning': '0.2.0'

Python version: 3.9.6 with ipython 7.20.0

Operating system: Windows 10 using WSL 2

What is the current behavior?
Using the .draw(output = 'mpl') for a QuantumCircuit object or the function plot_bloch_multivector results in two copies of the same picture even if %matplotlib inline has been disabled.
Steps to reproduce the problem
This is the code I run in one code block:
import qiskit
from qiskit import QuantumCircuit, execute, Aer
from qiskit.visualization import plot_bloch_multivector
import math

qc = QuantumCircuit(2)
qc.h(0)
qc.h(1)
qc.draw(output='mpl', scale=1)

followed by the next code block:
out_vector = execute(qc, Aer.get_backend('statevector_simulator')).result().get_statevector()
plot_bloch_multivector(out_vector, reverse_bits=True)

What is the expected behavior?
Should get only one single plot per code block
Suggested solutions
The suggested solution from https://github.com/Qiskit/qiskit/issues/1265 of removing %matplotlib inline does not work

Should hopefully have been fixed by Qiskit/qiskit-terra#6889.  Please feel free to re-open on the Qiskit Terra repo if that's not the case.


Remove dependency from Prop-types

Hi,
Please remove dependency from prop-types package. It is unnecessary making the product heavy.
Have a look at https://www.npmjs.com/package/babel-plugin-transform-react-remove-prop-types
Thanks,
Satish

Thanks. I'll look into this.


error generating code

Error when upload a screenshot, but no log found in backend and backend

When i close the backend, the same error happen. It seems that the backend do not receive the messege from the frontend

It works, Thank you


Update google-chrome to 63.0.3239.108

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Thanks for the quick turn around!


Repository Deprecated

I will stop supporting and implementing Docker images starting in early 2019.
Repositories will be transferred to blacklabelops-legacy and archived for read only purpose.
The following repositories and images are affected:
January 2019

blacklabelops/crowd
blacklabelops/nginx
blacklabelops/bitbucket
blacklabelops/java
blacklabelops/letsencrypt
blacklabelops/postgres

March 2019

blacklabelops/baseimages
blacklabelops/confluence
blacklabelops/jira
blacklabelops/volumerize
blacklabelops/logrotate

Why did you come to this decision? Are you not using Atlassian products anymore or did you switch to another image yourself?

Due to my career path. I am more involved in development than ops. I am not even involved with anything Docker related.
The images are state of the art but they cost maintenance time.
This had been fun but it's time to make place for something new.

Then I wish you the best for your future path. 👍
I have to work a bit on understanding docker better so I may can jump in and maintain my own images soon.
You did a great job with this it inspires a lot of people I guess. 🥇

Transfer complete


Keras layer weights/sublayers getting deleted when creating a model with them. model.summary() / plot_model still shows those weights as part of graph though

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google colab enviroment

TensorFlow installed from (source or binary): Google colab default

Python version: Python 3, Google colab default

CUDA/cuDNN version:   Google colab default

GPU model and memory:  Tested on both Google colab p-100 GPU and CPU

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:
2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

2020-06-20 21:44:17.003371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
v2.2.0-0-g2b96f3662b 2.2.0

Describe the current behavior
I created a new model using two layers from and old model. However, now all of the layers/weights from the old model are Not showing up in the new model.
model.summary() and
tf.keras.utils.plot_model(
    model, to_file='model.png', show_shapes=False, show_layer_names=True,
    rankdir='TB', expand_nested=False, dpi=96
)

still has those weights, so I think they're a part of the graph.  But when I print them out, those weights/layers are missing altogether
Describe the expected behavior
All weights from component layers to should be in the model.
Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
Here is a Colabnotebook with a minimal example that reproduced the issue.
https://colab.research.google.com/drive/1n3_XNhdgH6Qo7GT-M570lIKWAoU3TML5?usp=sharing
And here is the code
!pip install transformers --q
%tensorflow_version 2.x

from transformers import TFBertModel, AutoModel, TFRobertaModel, AutoTokenizer
import tensorflow as tf
import tensorflow_addons as tfa

tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

from tensorflow import keras
from tensorflow.keras import layers
from copy import deepcopy

logger = tf.get_logger()
logger.info(tf.__version__)

def get_mini_models():
    tempModel = TFRobertaModel.from_pretrained('bert-base-uncased', from_pt=True)

    layer9 = deepcopy(tempModel.layers[0].encoder.layer[8])
    layer10 = deepcopy(tempModel.layers[0].encoder.layer[9])

    inputHiddenVals = tf.keras.Input(shape=[None, None], dtype=tf.float32, name='input_Q',
                                    batch_size=None)

    hidden1 = layer9((inputHiddenVals, None, None))
    hidden2 = layer10((hidden1[0], None, None))
    modelNew = tf.keras.Model(inputs=inputHiddenVals, outputs=hidden2)

    del tempModel

    return modelNew

@tf.function
def loss_fn(_, probs):
    bs = tf.shape(probs)[0]
    labels = tf.eye(bs, bs)
    return tf.losses.categorical_crossentropy(labels,
                                              probs,
                                              from_logits=True)

model = get_mini_models()
model.compile(loss=loss_fn,
                optimizer=tfa.optimizers.AdamW(weight_decay=1e-4, learning_rate=1e-5,
                                                epsilon=1e-06))

# Get model and layers directly to compare
tempModel = TFRobertaModel.from_pretrained('bert-base-uncased', from_pt=True)
layer9 = deepcopy(tempModel.layers[0].encoder.layer[8])
layer10 = deepcopy(tempModel.layers[0].encoder.layer[9])

# Only one layer, and that layer also has missing weights.
for i, var in enumerate(model.weights):
    print(model.weights[i].name)

# Full weights for one layer
for i, var in enumerate(layer9.weights):
    print(layer9.weights[i].name)

# Test what correct output should be

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
inputt = tokenizer.encode('This is a sentence', return_tensors='tf')
outt = tempModel(inputt)[0]

# Test model output. Not the same.

model(outt)

# Model summary somehow lists the weights
model.summary()

# Model diagram shows the correct connections between all the layers.

tf.keras.utils.plot_model(
    model, to_file='model.png', show_shapes=False, show_layer_names=True,
    rankdir='TB', expand_nested=False, dpi=96
)

Edit: I also tried making the layers from scratch, and setting the weights directly, and got the same result. Here's a colab notebook that does this. https://colab.research.google.com/drive/1EC_fObSp9lUsj_PFaYgFtRI93ErPYmU9?usp=sharing
And here's the code
!pip install transformers --q
%tensorflow_version 2.x

from transformers import TFBertModel, AutoModel, TFRobertaModel, AutoTokenizer

import tensorflow as tf
import tensorflow_addons as tfa

tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.layers import (Dense,
                                     Dropout)
import numpy as np
import os

logger = tf.get_logger()
logger.info(tf.__version__)

class TFBertSelfAttention2(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        if config.hidden_size % config.num_attention_heads != 0:
            raise ValueError(
                "The hidden size (%d) is not a multiple of the number of attention "
                "heads (%d)" % (config.hidden_size, config.num_attention_heads)
            )

        self.num_attention_heads = config.num_attention_heads
        assert config.hidden_size % config.num_attention_heads == 0
        self.attention_head_size = int(config.hidden_size / config.num_attention_heads)
        self.all_head_size = self.num_attention_heads * self.attention_head_size

        self.query = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=get_initializer(config.initializer_range), name="query_2"
        )
        self.key = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=get_initializer(config.initializer_range), name="key_2"
        )
        self.value = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=get_initializer(config.initializer_range), name="value_2"
        )

        self.dropout = tf.keras.layers.Dropout(config.attention_probs_dropout_prob)

    def transpose_for_scores(self, x, batch_size):
        x = tf.reshape(x, (batch_size, -1, self.num_attention_heads, self.attention_head_size))
        return tf.transpose(x, perm=[0, 2, 1, 3])

    def call(self, inputs, training=False):
        hidden_states, attention_mask, head_mask, output_attentions = inputs

        batch_size = shape_list(hidden_states)[0]
        mixed_query_layer = self.query(hidden_states)
        mixed_key_layer = self.key(hidden_states)
        mixed_value_layer = self.value(hidden_states)

        query_layer = self.transpose_for_scores(mixed_query_layer, batch_size)
        key_layer = self.transpose_for_scores(mixed_key_layer, batch_size)
        value_layer = self.transpose_for_scores(mixed_value_layer, batch_size)

        # Take the dot product between "query" and "key" to get the raw attention scores.
        attention_scores = tf.matmul(
            query_layer, key_layer, transpose_b=True
        )  # (batch size, num_heads, seq_len_q, seq_len_k)
        dk = tf.cast(shape_list(key_layer)[-1], tf.float32)  # scale attention_scores
        attention_scores = attention_scores / tf.math.sqrt(dk)

        if attention_mask is not None:
            # Apply the attention mask is (precomputed for all layers in TFBertModel call() function)
            attention_scores = attention_scores + attention_mask

        # Normalize the attention scores to probabilities.
        attention_probs = tf.nn.softmax(attention_scores, axis=-1)

        # This is actually dropping out entire tokens to attend to, which might
        # seem a bit unusual, but is taken from the original Transformer paper.
        attention_probs = self.dropout(attention_probs, training=training)

        # Mask heads if we want to
        if head_mask is not None:
            attention_probs = attention_probs * head_mask

        context_layer = tf.matmul(attention_probs, value_layer)

        context_layer = tf.transpose(context_layer, perm=[0, 2, 1, 3])
        context_layer = tf.reshape(
            context_layer, (batch_size, -1, self.all_head_size)
        )  # (batch_size, seq_len_q, all_head_size)

        outputs = (
            (context_layer, attention_probs) if cast_bool_to_primitive(output_attentions) is True else (context_layer,)
        )

        return outputs

class TFBertSelfOutput2(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.dense = tf.keras.layers.Dense(
            config.hidden_size, kernel_initializer=get_initializer(config.initializer_range), name="dense2"
        )
        self.LayerNorm = tf.keras.layers.LayerNormalization(epsilon=config.layer_norm_eps, name="LayerNorm2")
        self.dropout = tf.keras.layers.Dropout(config.hidden_dropout_prob)

    def call(self, inputs, training=False):
        hidden_states, input_tensor = inputs

        hidden_states = self.dense(hidden_states)
        hidden_states = self.dropout(hidden_states, training=training)
        hidden_states = self.LayerNorm(hidden_states + input_tensor)
        return hidden_states

class TFBertAttention2(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.self_attention = TFBertSelfAttention2(config, name="self2")
        self.dense_output = TFBertSelfOutput2(config, name="output2")

    def prune_heads(self, heads):
        raise NotImplementedError

    def call(self, inputs, training=False):
        input_tensor, attention_mask, head_mask, output_attentions = inputs

        self_outputs = self.self_attention(
            [input_tensor, attention_mask, head_mask, output_attentions], training=training
        )
        attention_output = self.dense_output([self_outputs[0], input_tensor], training=training)
        outputs = (attention_output,) + self_outputs[1:]  # add attentions if we output them
        return outputs

class TFBertIntermediate2(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.dense = tf.keras.layers.Dense(
            config.intermediate_size, kernel_initializer=get_initializer(config.initializer_range), name="dense2"
        )
        if isinstance(config.hidden_act, str):
            self.intermediate_act_fn = ACT2FN[config.hidden_act]
        else:
            self.intermediate_act_fn = config.hidden_act

    def call(self, hidden_states):
        hidden_states = self.dense(hidden_states)
        hidden_states = self.intermediate_act_fn(hidden_states)
        return hidden_states

class TFBertOutput2(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.dense = tf.keras.layers.Dense(
            config.hidden_size, kernel_initializer=get_initializer(config.initializer_range), name="dense2"
        )
        self.LayerNorm = tf.keras.layers.LayerNormalization(epsilon=config.layer_norm_eps, name="LayerNorm2")
        self.dropout = tf.keras.layers.Dropout(config.hidden_dropout_prob)

    def call(self, inputs, training=False):
        hidden_states, input_tensor = inputs

        hidden_states = self.dense(hidden_states)
        hidden_states = self.dropout(hidden_states, training=training)
        hidden_states = self.LayerNorm(hidden_states + input_tensor)
        return hidden_states

class TFBertLayer2(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.attention = TFBertAttention2(config, name="attention2")
        self.intermediate = TFBertIntermediate2(config, name="intermediate2")
        self.bert_output = TFBertOutput2(config, name="output2")

    def call(self, inputs, training=False):
        hidden_states, attention_mask, head_mask, output_attentions = inputs

        attention_outputs = self.attention(
            [hidden_states, attention_mask, head_mask, output_attentions], training=training
        )
        attention_output = attention_outputs[0]
        intermediate_output = self.intermediate(attention_output)
        layer_output = self.bert_output([intermediate_output, attention_output], training=training)
        outputs = (layer_output,) + attention_outputs[1:]  # add attentions if we output them
        return outputs

class TFBertSelfAttention(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        if config.hidden_size % config.num_attention_heads != 0:
            raise ValueError(
                "The hidden size (%d) is not a multiple of the number of attention "
                "heads (%d)" % (config.hidden_size, config.num_attention_heads)
            )

        self.num_attention_heads = config.num_attention_heads
        assert config.hidden_size % config.num_attention_heads == 0
        self.attention_head_size = int(config.hidden_size / config.num_attention_heads)
        self.all_head_size = self.num_attention_heads * self.attention_head_size

        self.query = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=get_initializer(config.initializer_range), name="query_"
        )
        self.key = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=get_initializer(config.initializer_range), name="key_"
        )
        self.value = tf.keras.layers.Dense(
            self.all_head_size, kernel_initializer=get_initializer(config.initializer_range), name="value_"
        )

        self.dropout = tf.keras.layers.Dropout(config.attention_probs_dropout_prob)

    def transpose_for_scores(self, x, batch_size):
        x = tf.reshape(x, (batch_size, -1, self.num_attention_heads, self.attention_head_size))
        return tf.transpose(x, perm=[0, 2, 1, 3])

    def call(self, inputs, training=False):
        hidden_states, attention_mask, head_mask, output_attentions = inputs

        batch_size = shape_list(hidden_states)[0]
        mixed_query_layer = self.query(hidden_states)
        mixed_key_layer = self.key(hidden_states)
        mixed_value_layer = self.value(hidden_states)

        query_layer = self.transpose_for_scores(mixed_query_layer, batch_size)
        key_layer = self.transpose_for_scores(mixed_key_layer, batch_size)
        value_layer = self.transpose_for_scores(mixed_value_layer, batch_size)

        # Take the dot product between "query" and "key" to get the raw attention scores.
        attention_scores = tf.matmul(
            query_layer, key_layer, transpose_b=True
        )  # (batch size, num_heads, seq_len_q, seq_len_k)
        dk = tf.cast(shape_list(key_layer)[-1], tf.float32)  # scale attention_scores
        attention_scores = attention_scores / tf.math.sqrt(dk)

        if attention_mask is not None:
            # Apply the attention mask is (precomputed for all layers in TFBertModel call() function)
            attention_scores = attention_scores + attention_mask

        # Normalize the attention scores to probabilities.
        attention_probs = tf.nn.softmax(attention_scores, axis=-1)

        # This is actually dropping out entire tokens to attend to, which might
        # seem a bit unusual, but is taken from the original Transformer paper.
        attention_probs = self.dropout(attention_probs, training=training)

        # Mask heads if we want to
        if head_mask is not None:
            attention_probs = attention_probs * head_mask

        context_layer = tf.matmul(attention_probs, value_layer)

        context_layer = tf.transpose(context_layer, perm=[0, 2, 1, 3])
        context_layer = tf.reshape(
            context_layer, (batch_size, -1, self.all_head_size)
        )  # (batch_size, seq_len_q, all_head_size)

        outputs = (
            (context_layer, attention_probs) if cast_bool_to_primitive(output_attentions) is True else (context_layer,)
        )

        return outputs

class TFBertSelfOutput(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.dense = tf.keras.layers.Dense(
            config.hidden_size, kernel_initializer=get_initializer(config.initializer_range), name="dense"
        )
        self.LayerNorm = tf.keras.layers.LayerNormalization(epsilon=config.layer_norm_eps, name="LayerNorm")
        self.dropout = tf.keras.layers.Dropout(config.hidden_dropout_prob)

    def call(self, inputs, training=False):
        hidden_states, input_tensor = inputs

        hidden_states = self.dense(hidden_states)
        hidden_states = self.dropout(hidden_states, training=training)
        hidden_states = self.LayerNorm(hidden_states + input_tensor)
        return hidden_states

class TFBertAttention(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.self_attention = TFBertSelfAttention(config, name="self")
        self.dense_output = TFBertSelfOutput(config, name="output")

    def prune_heads(self, heads):
        raise NotImplementedError

    def call(self, inputs, training=False):
        input_tensor, attention_mask, head_mask, output_attentions = inputs

        self_outputs = self.self_attention(
            [input_tensor, attention_mask, head_mask, output_attentions], training=training
        )
        attention_output = self.dense_output([self_outputs[0], input_tensor], training=training)
        outputs = (attention_output,) + self_outputs[1:]  # add attentions if we output them
        return outputs

class TFBertIntermediate(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.dense = tf.keras.layers.Dense(
            config.intermediate_size, kernel_initializer=get_initializer(config.initializer_range), name="dense"
        )
        if isinstance(config.hidden_act, str):
            self.intermediate_act_fn = ACT2FN[config.hidden_act]
        else:
            self.intermediate_act_fn = config.hidden_act

    def call(self, hidden_states):
        hidden_states = self.dense(hidden_states)
        hidden_states = self.intermediate_act_fn(hidden_states)
        return hidden_states

class TFBertOutput(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.dense = tf.keras.layers.Dense(
            config.hidden_size, kernel_initializer=get_initializer(config.initializer_range), name="dense"
        )
        self.LayerNorm = tf.keras.layers.LayerNormalization(epsilon=config.layer_norm_eps, name="LayerNorm")
        self.dropout = tf.keras.layers.Dropout(config.hidden_dropout_prob)

    def call(self, inputs, training=False):
        hidden_states, input_tensor = inputs

        hidden_states = self.dense(hidden_states)
        hidden_states = self.dropout(hidden_states, training=training)
        hidden_states = self.LayerNorm(hidden_states + input_tensor)
        return hidden_states

class TFBertLayer(tf.keras.layers.Layer):
    def __init__(self, config, **kwargs):
        super().__init__(**kwargs)
        self.attention = TFBertAttention(config, name="attention")
        self.intermediate = TFBertIntermediate(config, name="intermediate")
        self.bert_output = TFBertOutput(config, name="output")

    def call(self, inputs, training=False):
        hidden_states, attention_mask, head_mask, output_attentions = inputs

        attention_outputs = self.attention(
            [hidden_states, attention_mask, head_mask, output_attentions], training=training
        )
        attention_output = attention_outputs[0]
        intermediate_output = self.intermediate(attention_output)
        layer_output = self.bert_output([intermediate_output, attention_output], training=training)
        outputs = (layer_output,) + attention_outputs[1:]  # add attentions if we output them
        return outputs

configBase = {
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": 0,
  "eos_token_id": 2,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-05,
  "max_position_embeddings": 514,
  "model_type": "roberta",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 1,
  "type_vocab_size": 1,
  "vocab_size": 50265
}

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

config = AttrDict(configBase)

def get_initializer(initializer_range=0.02):
    """Creates a `tf.initializers.truncated_normal` with the given range.
    Args:
        initializer_range: float, initializer range for stddev.
    Returns:
        TruncatedNormal initializer with stddev = `initializer_range`.
    """
    return tf.keras.initializers.TruncatedNormal(stddev=initializer_range)

def gelu(x):
    """ Gaussian Error Linear Unit.
    Original Implementation of the gelu activation function in Google Bert repo when initially created.
        For information: OpenAI GPT's gelu is slightly different (and gives slightly different results):
        0.5 * x * (1 + torch.tanh(math.sqrt(2 / math.pi) * (x + 0.044715 * torch.pow(x, 3))))
        Also see https://arxiv.org/abs/1606.08415
    """
    cdf = 0.5 * (1.0 + tf.math.erf(x / tf.math.sqrt(2.0)))
    return x * cdf

ACT2FN = {
    "gelu": tf.keras.layers.Activation(gelu),
}

def shape_list(x):
    """Deal with dynamic shape in tensorflow cleanly."""
    static = x.shape.as_list()
    dynamic = tf.shape(x)
    return [dynamic[i] if s is None else s for i, s in enumerate(static)]

def cast_bool_to_primitive(bool_variable, default_tensor_to_true=False):
    """Function arguments can be inserted as boolean tensor
        and bool variables to cope with keras serialization
        we need to cast `output_attentions` to correct bool
        if it is a tensor
    Args:
        default_tensor_to_true: bool, if tensor should default to True
        in case tensor has no numpy attribute
    """
    # if bool variable is tensor and has numpy value
    if tf.is_tensor(bool_variable):
        if hasattr(bool_variable, "numpy"):
            return bool(bool_variable.numpy())
        elif default_tensor_to_true:
            return True

    # else variable is bool
    return bool_variable

def get_2_transformerLayerP(numb):
    tokenizer = AutoTokenizer.from_pretrained('allenai/biomed_roberta_base')
    inputt = tokenizer.encode('This is a sentence', return_tensors='tf')
    tempModel = TFRobertaModel.from_pretrained('allenai/biomed_roberta_base', from_pt=True)
    outt = tempModel(inputt)[0]

    t_layer11 = TFBertLayer(config, name="layer_._{}".format(11+numb))
    t_layer12 = TFBertLayer2(config, name="layer_._{}".format(12+numb))

    t_layer11((outt, None, None, None))
    t_layer12((outt, None, None, None))

    t_layer11.set_weights( tempModel.layers[0].encoder.layer[10].get_weights() )
    t_layer12.set_weights( tempModel.layers[0].encoder.layer[11].get_weights() )

    t_layer12.intermediate.intermediate_act_fn = tf.keras.activations.tanh

    del tokenizer
    del tempModel

    return t_layer11, t_layer12

def get_mini_models():
    P_trans11, P_trans12 = get_2_transformerLayerP(6)

    inputHiddenVals = tf.keras.Input(shape=[None, None], dtype=tf.float32, name='input_Q',
                                    batch_size=None)

    P_outputs = P_trans11((inputHiddenVals, None, None, None))[0]
    P_outputsFinal = P_trans12((P_outputs, None, None, None))[0]
    modelNew = tf.keras.Model(inputs=inputHiddenVals, outputs=P_outputsFinal)

    return modelNew

@tf.function
def loss_fn(_, probs):

    bs = tf.shape(probs)[0]
    labels = tf.eye(bs, bs)
    return tf.losses.categorical_crossentropy(labels,
                                              probs,
                                              from_logits=True)

model = get_mini_models()
model.compile(loss=loss_fn,
                optimizer=tfa.optimizers.AdamW(weight_decay=1e-4, learning_rate=1e-5,
                                                epsilon=1e-06))

for i, var in enumerate(model.trainable_weights):
    print(model.trainable_weights[i].name)

@Santosh-Gupta
I have tried in colab with TF version 2.2 .Please, find the gist here.Is this the expected behavior?.Thanks!

Hey

@Santosh-Gupta
I have tried in colab with TF version 2.2 .Please, find the gist here.Is this the expected behavior?.Thanks!

This was a little tricky to checkout, so I split up the cells so some of the outputs are viewable
https://colab.research.google.com/gist/ravikyram/0191e12b7c6d9afeb80ccc009870b255/untitled52.ipynb
The output of cell 6 is
tf_roberta_model_1/roberta/encoder/layer_._8/attention/self/query/kernel:0
tf_roberta_model_1/roberta/encoder/layer_._8/attention/self/query/bias:0
tf_roberta_model_1/roberta/encoder/layer_._8/attention/self/key/kernel:0
tf_roberta_model_1/roberta/encoder/layer_._8/attention/self/key/bias:0
tf_roberta_model_1/roberta/encoder/layer_._8/attention/self/value/kernel:0
tf_roberta_model_1/roberta/encoder/layer_._8/attention/self/value/bias:0

which contains all the model weights. There are weights defined in the model that are missing.
It should look something like the output of cell 7, which prints out the weights of both of the layers which comprise the model.
tf_roberta_model_2/roberta/encoder/layer_._8/attention/self/query/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/self/query/bias:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/self/key/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/self/key/bias:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/self/value/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/self/value/bias:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/output/dense/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/output/dense/bias:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/output/LayerNorm/gamma:0
tf_roberta_model_2/roberta/encoder/layer_._8/attention/output/LayerNorm/beta:0
tf_roberta_model_2/roberta/encoder/layer_._8/intermediate/dense/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._8/intermediate/dense/bias:0
tf_roberta_model_2/roberta/encoder/layer_._8/output/dense/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._8/output/dense/bias:0
tf_roberta_model_2/roberta/encoder/layer_._8/output/LayerNorm/gamma:0
tf_roberta_model_2/roberta/encoder/layer_._8/output/LayerNorm/beta:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/self/query/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/self/query/bias:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/self/key/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/self/key/bias:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/self/value/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/self/value/bias:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/output/dense/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/output/dense/bias:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/output/LayerNorm/gamma:0
tf_roberta_model_2/roberta/encoder/layer_._9/attention/output/LayerNorm/beta:0
tf_roberta_model_2/roberta/encoder/layer_._9/intermediate/dense/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._9/intermediate/dense/bias:0
tf_roberta_model_2/roberta/encoder/layer_._9/output/dense/kernel:0
tf_roberta_model_2/roberta/encoder/layer_._9/output/dense/bias:0
tf_roberta_model_2/roberta/encoder/layer_._9/output/LayerNorm/gamma:0
tf_roberta_model_2/roberta/encoder/layer_._9/output/LayerNorm/beta:0

So one layer is missing altogether. And in the other layer, weights are missing as well.

The missing weights issue doesn't seem to happen if the architecture is made using keras model subclassing.
I made a colab gist showing the architecture made using model subclassing, and the same architecture made using the functional API, with the latter resulting in missing trainable weights.
https://colab.research.google.com/gist/Santosh-Gupta/273361f873e4daf572fddea691b1f325/missingtrainablevars.ipynb
The gist uses the huggingface transformers library to create the layers, so the code is clearer to look through, though it doesn't show the layer code like in my previous colab notebook.
Another issue; using model subclassing may not be a viable workaround for TPU training my architecture; It looks like TPU training does not support subclassed model, as described in this stackoverflow post https://stackoverflow.com/questions/60444486/use-tf-distribute-strategies-with-tf-keras-model-subclassing

ValueError: We currently do not support distribution strategy with a Sequential model that is created without input_shape/input_dim set in its first layer or a subclassed model.

I am also running into a different issue with the model, which is described in a different github issue (not sure if its related to model subclassing)
https://github.com/tensorflow/tensorflow/issues/41074

I tried both the nightly version of tf 2.2.0 and the tf 2.3.0-rc1 in case there was some sort of fix, but both resulted in missing trainable variables for using the functional API.
For convenience, here is a github gist of the code I ran.
https://colab.research.google.com/gist/Santosh-Gupta/b02668b47744655ce32e6d427357cb35/missingtrainablevars_nightly_2-3-0-rc1.ipynb

I wanted to check if the inference was the same for both models, and they are. I tested it using both the Transformers library and creating each layer from scratch (Transformers library only used to copy/set consistent weights)
Here are the colab gists
https://colab.research.google.com/gist/Santosh-Gupta/ec9f7c8a189a8d99e73d96fbe728aef8/model_weight_debug_scratch_public_inference.ipynb
https://colab.research.google.com/gist/Santosh-Gupta/766a27c1500a330cba6f479805dad27d/missingtrainablevarsinference.ipynb
I sort of went through the from scratch version in details, and I am unable to figure out why this is happening for the functional api. From my understanding if the Keras Model fit function, if the weights aren't in the list of the trainable_variables, then they won't receive gradient updates. And this issue seems prone to any Keras model that uses custom layers with the functional API.
Is this something anyone using custom Keras layers with the Functional API should be worried about?

Hi @Santosh-Gupta:
Will reply w/ a longer explanation shortly, but it looks like what's going on is:
The layers currently enter a 'functional api construction' mode only if all of the inputs in the first argument come from other Keras layers. However, you have None included in the inputs in the first positional arg, so it's not triggering functional api construction.
That causes the layer to get 'inlined' in the outer functional model rather than correctly included. You should be able to work around this by changing the layer api so Nones should not get passed in.
We have a major cleanup/refactoring of the Functional API mostly done that will fix this & a number of other issues w/ it. But, that will only land in 2.4. It's not immediately obvious if we can squeeze a fix into tf 2.3 as the RC is already out.

Thanks for the reply! I am wondering what you mean by 'outer functional model', I tried googling 'keras outer functional model' and 'keras outer model' but I wasn't able to find anything.
Are we sure that the Nones being included inputs is the cause of missing variables? I am wondering if I should close the issue.

Ah sorry by outer functional model I just meant the functional model you are building (as opposed to the layers inside of the model).
Essentially rather than constructing the functional model as:
inputs -> layer 1 -> layer 2 -> outputs
(which is what should be expected),
it ends up inlining all the contents of layer 1 & layer 2:
inputs -> first op in layer 1 -> nested sublayer in layer 1 -> etc. -> first op in layer 2 ... -> outputs
And so the model lacks a reference to layer 1 & layer 2 themselves and misses any weights that weren't contained in their subweights.

For clarity, should sending tuples/lists be avoided when sending to layers, or just Nones?
Arbitrary data structures should be fine (as should data structures that may or may not contain symbolic values in positional/keyword args outside of the first arg)

What you're experiencing is specifically an issue when whatever data structure is passed to the first positional arg happens to contain any item that is not a symbolic functional input/output.
This specific behavior is a historical edge case dating back to when Keras layers only ever accepted a single positional argument that could not be an arbitrary data structure, and all of the inputs had to be symbolic keras inputs/outputs. Unfortunately it's caused this surprising behavior when combined w/ other functionality that has been added since (automatically turning tf op layers into keras layers).
So, historically trying to pass in Nones like you're doing would have triggered a (hard to interpret) error message, because TF/Keras wouldn't be able to inline the tf ops inside the functional model when it calls the layer. Now it silently behaves in a way you didn't expect because tf ops can be used during functional API construction.

I am wondering if a possible workaround is either a layer or model that can append Nones to the inputs/outputs.
So something like
class transformer_IO(tf.keras.layers.Layer):
    def call(self, input):
        return (input, None, None, None)

or
class transformer_IO_model(tf.keras.Model):
    def call(self, input):
        return (input, None, None, None)

Using either as-is results in "AttributeError: 'NoneType' object has no attribute 'shape'". I am wondering if this sort of workaround is worth further exploring, or if I should just work on redesigning the layers.
For convenience, here is a colab gist of my attempts
https://colab.research.google.com/gist/Santosh-Gupta/3b5a2c6bc288c58eda992e08da1986ee/workaroundattempt.ipynb

Layers in a functional API can only output tensors/data structures of tensors.
As a workaround in the same vein though if you don't want to redesign the underlying layers:
You can have a layer contain a nested transformer layer, and pass the input args + Nones to the nested layer. You can then use this layer in a functional model w/o issues.
Alternatively if you're okay trying the tf nightlies, you can experiment with the refactoring I mentioned earlier by directly flipping our internal experimental flag:
from tensorflow.python.keras.engine import keras_tensor
keras_tensor.enable_keras_tensors()

I believe it should fix your issue and it should make the functional api generally much more reliable, but like I said it will only be landing in 2.4.

from tensorflow.python.keras.engine import keras_tensor
keras_tensor.enable_keras_tensors()

This solved the issue, at least all the weight show up now. For people checking in on this, I used tf-nightly 2.4.0.dev20200715 (just in case it breaks in a future version of nightly)
I'll be testing out training soon.

Update on this issue: We weren't able to get a fix into 2.3 for a variety of reasons, but we were able to add raising a meaningful error message for 2.3 in this setting rather than silently missing some of the weights.
(The proper fix is still in the nightlies guarded by the Functional API refactoring, as mentioned above.)

The functional API refactoring has landed in the nightlies, so this should now be fixed in the nightlies.


PD - Balanced not listed as an options

Hi,
From below:
terraform/modules/hana-scaleup/locals.tf
Looks like there's no PD-Balanced supported for deployment
but from documentation PD-Balanced supported with certain sizes:
https://cloud.google.com/solutions/sap/docs/sap-hana-planning-guide#minimum_sizes_for_ssd-based_persistent_disks
Could you please kindly advise whether PD-Balanced will be supported for deployment?
Thanks - Budi

@busetde PD-Balanced is now supported for HANA.


Fix Fibonacci and Lucas number exercises

Fixes Fibonacci exercise

I had two separate pull requests for these two fixes, but I used the same branch for both fixes. That's why I removed the redundant one.


Deploy docs to GitHub Pages

Summary
This PR changes the deploy-docs.yml workflow to deploy the documentation to GitHub Pages to migrate the project away from the qiskit.org domain.
The deployment to the gh-pages branch will not overwrite the locale folder, where I will save all the translations in a follow-up once the first deployment is done.
More information about the change and a quick guide to set up GitHub pages can be found at: https://github.com/Qiskit/ecosystem/issues/578
Reminder task for maintainers here to update the doc link in the About text for the repo on the main page once this change happens and the docs are re-deployed.

[ ] Update About text doc link

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 75.822%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
646

Relevant Lines:
852

💛  - Coveralls

@arnaucasau With the deploy translatable strings script deleted I imagine the gettext section of the tox.ini file can be removed (the script used tox to do the strings). Also there is gettext config in the conf.py such as gettext_compact that is not really needed.
While you are having locales folder there not cleaned during deploy it seems that any further updates of the translations are not going to happen - hence the removal of the script.
I figured to sort things out here, since the other apps (Nature, ML and Opt) should be pretty much identical in regards this change. Once it all seems fine then backport this PR and deploy the docs to make sure its all as expected.

Thank you @woodsp-ibm for your review! In the last commit, I removed the gettext section from the tox.ini and the config related to it that I found in conf.py (I think that's everything).
I'll make the same changes to Nature, Machine learning, and Optimization :+1:


Fluid type scale

Add fluid type scales

Related: https://github.com/enhance-dev/enhance-styles/issues/14


How to cancel Jobs in Ecto Queue?

Hi,
Right now We are implementing honeydew jobs using ecto queue. We cannot able to find the right function to cancel jobs in ecto queue. There are functions to cancel Mnesia and erlang queue jobs. But now function is available on Ecto Queue. Can you Share an example function to cancel ecto jobs.

Hey @mohandass-pat,
Check out the Ecto queue example tests:
https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/test/ecto_poll_queue_example_test.exs#L86
Keep in mind that once a job is being run by a worker, there's no way to forcefully kill it at present. This function simply removes the job from the queue.


Set-SPOTheme gives file not found on subweb

Reporting an Issue or Missing Feature
As of a recent update, Set-SPOTheme no longer works for subsites, it reports file not found for the theme file precent in its parent's theme library. Set-SPOTheme works properly for the rootweb
Expected behavior
No file not found error, it properly sets the theme like it used to
Actual behavior
Set-SPOTheme : File Not Found.
At line:1 char:2

Set-SPOTheme -ColorPaletteUrl "https://test.sharepoint.com/teams/Test ...

  + CategoryInfo          : WriteError: (:) [Set-SPOTheme], ServerException
  + FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.Branding.SetTheme

Steps to reproduce behavior
Run the following code:
$colorFile = "$PSScriptRoot/theme/theme.spcolor"
$colorFileName = "theme.spcolor"
Connect-SPOnline -Url $DestinationsiteUrl -Credentials $Credentials
$web = Get-SPOWeb
$context = Get-SPOContext
Add-SPOFile -Path $colorFile -Folder "/_catalogs/theme/15/"
$colorFileUrl = $web.ServerRelativeUrl + "/_catalogs/theme/15/" + $colorFileName
Set-SPOTheme -ColorPaletteUrl $colorFileUrl
$webs = Get-SPOSubWebs -Recurse
foreach($subWeb In $webs){
$string = "Setting Theme and Logo of " + $subWeb.ServerRelativeUrl
Log $string
Set-SPOTheme -ColorPaletteUrl $colorFileUrl -Web $subWeb
}
Code ran fine in version 2.6.1608.0, in the latest version, the second Set-SPOTheme gives an error
Which version of the PnP-PowerShell Cmdlets are you using?

[ ] PnP PowerShell for SharePoint Online

What is the version of the Cmdlet module you are running?
2.8.1610.1
How did you install the PnP-PowerShell Cmdlets?

[ ] MSI Installed downloaded from GitHub

This is because the CSOM SetTheme functionality (or rather, the server) expects the Theme file (*.spcolor) to reside in the web you're applying it to. So if you upload the file to each subweb in the loop, it will work.
I'm about to update the cmdlet with two additional parameters. After we released that update you can execute the snippet above with one single cmdlet call:
Set-SPOTheme -ColorPaletterUrl $colorFileUrl -ResetSubwebsToInherit

This will set the theme on the rootweb and mark all subwebs to use the theme from the rootweb.
Resetting the theme to the default theme for the rootweb and all subwebs is a matter of
Set-SPOTheme -ResetSubwebsToInherit

Until that time, change your script to use:
foreach($subWeb in $webs) {
    Add-SPOFile -Path .\custom.spcolor -Folder "/_catalogs/theme/15" -Web $subWeb
    Set-PnPTheme -ColorPaletteUrl "$($subWeb.ServerRelativeUrl)/_catalogs/theme/15/custom.spcolor" -Web $subWeb
}

I just merged #543  which implements the additional parameters. They will become available in the November 2016 release.


release 1.1.6

Codecov Report

Merging #149 (1aa23e5) into master (74ce627) will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##              master      #149   +/-   ##
===========================================
  Coverage     36.351%   36.351%
  Complexity      1037      1037
===========================================
  Files            209       209
  Lines           9620      9620
  Branches        2389      2389
===========================================
  Hits            3497      3497
  Misses          5352      5352
  Partials         771       771

Flag
Coverage Δ

unittests
36.351% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 74ce627...1aa23e5. Read the comment docs.


Redudant testing that was creating issues for the CI-docker build

👍  from @josh-cogdill (under coercion)

No code change apart from remove of redundant tests so I'm merging this


[FEATURE] Scroll to top feature .

As I can see the website is under construction ,  as much as the components will increase scrolling will be required. It will be fair to add a scroll to top button , so that where ever the person is at , can directly scroll to top which will make it user friendly.
If this new feature is required do assign it to me under GSSOC 2023.
Changes will be done in no time. ! : |)

Hey @ManishaSwain8 , first we are implementing our design i.e. blueprint of website and secondly you can create the "scroll to top button" Component in the component folder and then we will import in App.jsx file
SO you can go ahead

Oh okay ..on it !

Hey @ManishaSwain8 , now you can start the work
make a beautiful scroll to top button only for home page

Sure !

Which library ?


Click not working, no error reported

Hello,
I try to follow this tutorial from Dan Bader on click but for some reason the code from the tuto (very basic) doesn't work in the command line with $ python cli.py 'London' and unfortunnatly it returns no error so it's difficult to investigate what is going on here.
However, the function current_weather() works like a charm in Spyder IDE, so initially I suspected a compatibility issue between Python (the Anaconda version) and the click module so I completly deinstalled Anaconda and I'm now running on python 3.6.7 for Ubuntu.
But still I'm unable to make it works in the CLI and it returns no error. What am'I doing wrong here?
import click
import requests

SAMPLE_API_KEY = 'b1b15e88fa797225412429c1c50c122a1'

@click.command()
@click.argument('location')
def main(location, api_key):
    weather = current_weather(location)
    print(f"The weather in {location} right now: {weather}.")

def current_weather(location, api_key=SAMPLE_API_KEY):
    url = 'http://samples.openweathermap.org/data/2.5/weather'

    query_params = {
        'q': location,
        'appid': api_key,
    }

    response = requests.get(url, params=query_params)

    return response.json()['weather'][0]['description']

In CLI:
$ python cli.py
$
$ python cli.py 'London'
$

In Spyder IDE:
In [1109]: location = 'London'

In [1110]: current_weather(location)
Out[1110]: 'light intensity drizzle'

When using with pdb source code debugger, pdb automatically enter post-mortem debugging which means the program being exits abnormally. But there is no error...
$ python -m pdb cli.py 'London'
> /home/project/cli.py(2)<module>()
-> import click
(Pdb)

I have click-7.0 installed and Python 3.6.7 (default, Oct 22 2018, 11:32:17)

Version 7.0
$ pip3 freeze | grep Click
Click==7.0

main has an unused api_key parameter. Also, the file doesn't actually start a cli, it just defines some things and exits.
if __name__ == "__main__":
    main()

This is documented.
Please use Stack Overflow for questions about your own code. This tracker is for issues related to the project itself. Be sure to include a minimal, complete, and verifiable example.

Thank you


Why is z-index=1010 for splitbar?

Hi,
I am having some issues with the split-bar being above some other elements. I am wondering why the z-index on the split bar is 1010 and what are the consequences of setting it to 0?
Thanks

It's set to such a high number so other elements don't appear on top of it.  The assumption is that the developer would want it at the top.  You are free to override this as you wish.


Support campaign games in h3viewer

Verify and test that campaign games can be correctly read and visualized in h3viewer.

Support added in https://github.com/oysteinmyrmo/homm3tools/commit/b10cee3cec410c1c3a3c6a67ab404b6d95cfa806


Unable to load shared library 'tensorflow' or one of its dependencies

System.DllNotFoundException: Unable to load shared library 'tensorflow' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtensorflow: cannot open shared object file: No such file or directory
at Tensorflow.c_api.TF_NewGraph()
at Tensorflow.Graph..ctor()
at Microsoft.ML.TensorFlow.TensorFlowUtils.LoadTFSession(IExceptionContext ectx, Byte[] modelBytes, String modelFile)
at Microsoft.ML.Transforms.TensorFlowTransformer.Create(IHostEnvironment env, ModelLoadContext ctx)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)

Please install tensorlfow native library from nuget.


Instantiate anonymous types less

Fixes issue: #7097
This operates during instantiateType to skip instantiation of anonymous types whose declarations are not bound by the type mapper that will do the instantiation.
I'm not sure this is the right approach, but I want to get suggestions on the right way to do it so I thought a strawman would help the discussion.

Weird. The linter failed on protocol.d.t.s, which this PR doesn't touch. But only on the push CI build, not the PR one.

It's possible you'll need to merge in from master to sync up with some unrelated changes.

Are there any specific speed changes you see when compiling tsc with the new version?

I haven't looked, except to test on all the RWC projects -- angular2 and bluebird.d.ts both ran out of memory before and now run just fine. But there should be plenty of unneeded instantiations that are no longer happening. I'll try to run some perf tests later.

@sandersn Just merged #7448 so I think we can close this one.


[Question]: Modify Array of Objects

Hi All,
I am new to dexie and I am having trouble to update one of my records in table.
Lets say I have this array of Objects {   profiles: [     {       id: '123456',     },     {       id: '654321',     }   ] } how can i wrap this to db.where(id).modify()
Appreciate the help!

But it seems like i found a solution, I query base on index id and from modify function it will return the list of datas where I use es6 find to match the corresponding id. Will close it now. Thanks!


Editor Application Not Responding After Script Save

My Unity application freezes and does not recover from the "Not Responding" state in task manager after saving scripts and unity re compiles.
This happened twice and I am unsure the cause but in both cases the only noticeable thing in common with both instances was that I had an inspector window displaying a struct that I have made.
The struct contains various properties that all manipulate 3 floats in the background in different ways. While it works changing one property value in the inspector also changes at least 80% of the other property values displayed.
Perhaps it is an unexpected infinite loop during serialization because of structure of my struct?

Closing, no good solution in general for this issue.


Bearing save method only saves the coefficients

Currently, using the MagneticBearing class as an example, we have:
>>> import ross as rs
>>> n = 0
>>> g0 = 1e-3
>>> i0 = 1.0
>>> ag = 1e-4
>>> nw = 200
>>> alpha = 0.392
>>> kp_pid = 1.0
>>> kd_pid = 1.0
>>> k_amp = 1.0
>>> k_sense = 1.0
>>> tag = "magneticbearing"
>>> mbearing = rs.MagneticBearingElement(n=n, g0=g0, i0=i0, ag=ag, nw=nw,alpha=alpha,
...                                   kp_pid=kp_pid, kd_pid=kd_pid, k_amp=k_amp,
...
>>> mbearing.save('mbearing.toml')

This will produce the following within a .toml file:
[MagneticBearingElement_None]
kxx = [ "-4640.623377181318",]
kyy = [ "-4640.623377181318",]
kxy = [ 0.0,]
kyx = [ 0.0,]
cxx = [ "4.645268645827145",]
cyy = [ "4.645268645827145",]
cxy = [ 0.0,]
cyx = [ 0.0,]
n = 0
color = "#355d7a"
scale_factor = 1

And we lose all the information going to variables such as g0, i0 etc.

Ideally, when loading a bearing we would like to have all this info about additional attributes but also avoid having to run expensive computation that have been already carried out before saving the bearing.
A possible solution would be to have run() methods within the bearings that have expensive calculation and an argument run=bool that could be False when loading the bearing from a file.


Warning message for deprecated yaml.load() call

/usr0/home/dskatz/.local/share/virtualenvs/overhead-timing-effects-in-ROS-Wl9qFQgJ/lib/python3.6/site-packages/roswire/proxy/service.py:71: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
d = yaml.load(output)

Kinda weird that GitHub didn't automatically close this issue. Anyway, this should be fixed.


feat(c++): property filtering provides an API that returns a VerticesCollection

Describe the enhancement requested
Our current property filtering is done at the VertexPropertyArrowChunkReader  level using enhanced Acero's expressions. The result is a chunk that holds the vertex property information rather than the vertex itself, can we provide an interface that returns a VerticesCollection, which is  similar to label filtering (https://github.com/apache/incubator-graphar/pull/607) for subsequent operations.
Component(s)
C++

Please provide some suggestions. @acezen


Have the polling place loader, and DEV, stop sending things to Sentry

AB#138

Closing and merging into https://github.com/keithamoss/demsausage/issues/1339


cannot unpack non-iterable bool object

getUserFeed  not working any more
using getUserFeed example get that error
TypeError: cannot unpack non-iterable bool object

Bug fixed

yes now it work, however there  is another problem
if the user id is not English for example french (have accent) it will not work...
for example: user id "julméré"
any idea?

you can send me link tiktok user

you can send me link tiktok user

https://www.tiktok.com/@julméré

Updated, you can you use  Api.getUserFeed(user_url='https://www.tiktok.com/@MS4wLjABAAAAZjjeyobMEXLT9jVMWG9Su8ZuGVLWONB-VlbV1oyjwVFrz8RwRMR37ze_GlyO7ROR')

it's working very well, thanks alot


QuickPick doesn't fire the onDidChangeValue when setting value manually

If you set QuickPick.value programatically when setting up the QuickPick, it fails to trigger the onDidChangeValue event.
Here is my workaround hack:
https://github.com/eamodio/vscode-gitlens/blob/db5285d4ad268148dbc4a6ce9d1f284d5e18ac4e/src/commands/gitCommands.ts#L717-L727

verification steps:
Add this to an extension:
    const quickpick = window.createQuickPick();
    quickpick.title = 'Hello fam'
    quickpick.items = [{ label: "don't pick me" }, { label: 'or me' }]

    quickpick.onDidChangeValue((e) => window.showInformationMessage(e));
    quickpick.buttons = [
        {
            iconPath: ThemeIcon.File,
            tooltip: 'change value'
        }
    ];
    quickpick.onDidTriggerButton(e => {
        quickpick.value = "Changed!";
    });

    quickpick.show();

Type a couple characters in the filter, it should spam notifications
Click the button, it should change the value and send a notification with "Changed!".

@stevenguh, thanks for working around this change. Just for some explanation, we did this to align .value with .items and .selectedItems. At the end of the day, these properties now all have the same behavior which I think will be easier going forward.

Thanks for following up. That makes sense if the angle is to align with other onChange methods. Still it would be nice to have onChange methods that don't trigger on setter. That's probably a separate issue at this point.

verification steps above: https://github.com/microsoft/vscode/issues/122939#issuecomment-832201714


az signalr upstream update fails when the target has a location policy

Describe the bug
Looks to have been introduced with #27542. An empty location string is added to the request body being sent to  'https://management.azure.com/subscriptions/<...>/resourceGroups/<...>/providers/Microsoft.SignalRService/signalR/<...>?api-version=2023-06-01-preview'
{
  "location": "",
  "properties": {
    "upstream": {
      "templates": [
        { "urlTemplate": "https://<redacted>/runtime/webhooks/signalr?code=***" }
      ]
    },
    "publicNetworkAccess": "Enabled",
    "disableLocalAuth": false,
    "disableAadAuth": false
  }
}

If you also have a location policy on the target resource, the request will fail and also the command.
Related command
az signalr upstream update
Errors
Code: RequestDisallowedByPolicy
Message: Resource '' was disallowed by policy. Policy identifiers: '[{"policyAssignment":{"name":"Allowed locations","id":"/providers/Microsoft.Management/managementGroups//providers/Microsoft.Authorization/policyAssignments/"},"policyDefinition":{"name":"Allowed locations","id":"/providers/Microsoft.Authorization/policyDefinitions/"}}]'.
Target:
Additional Information:Type: PolicyViolation
Info:
{
    "evaluationDetails": {
        "evaluatedExpressions": [
            {
                "result": "True",
                "expressionKind": "Field",
                "expression": "location",
                "path": "location",
                "expressionValue": "",
                "targetValue": [
                    "westeurope",
                    "southcentralus",
                    "southeastasia",
                    "northeurope",
                    "northcentralus",
                    "eastasia"
                ],
                "operator": "NotIn"
            },
            {
                "result": "True",
                "expressionKind": "Field",
                "expression": "location",
                "path": "location",
                "expressionValue": "",
                "targetValue": "global",
                "operator": "NotEquals"
            },
            {
                "result": "True",
                "expressionKind": "Field",
                "expression": "type",
                "path": "type",
                "expressionValue": "Microsoft.SignalRService/signalR",
                "targetValue": "Microsoft.AzureActiveDirectory/b2cDirectories",
                "operator": "NotEquals"
            }
        ]
    },
    "policyDefinitionId": "/providers/Microsoft.Authorization/policyDefinitions/<redacted>",
    "policyDefinitionName": "<redacted>",
    "policyDefinitionDisplayName": "Allowed locations",
    "policyDefinitionEffect": "deny",
    "policyAssignmentId": "/providers/Microsoft.Management/managementGroups/<redacted>/providers/Microsoft.Authorization/policyAssignments/09a<redacted>",
    "policyAssignmentName": "<redacted>",
    "policyAssignmentDisplayName": "Allowed locations",
    "policyAssignmentScope": "/providers/Microsoft.Management/managementGroups/<redacted>",
    "policyAssignmentParameters": {
        "listOfAllowedLocations": [
            "westeurope",
            "southcentralus",
            "southeastasia",
            "northeurope",
            "northcentralus",
            "eastasia"
        ]
    },
    "policyExemptionIds": []
}

Issue script & Debug output
Heavily redacted for obvious reasons, but it contains the key information:
az signalr upstream update --name <...> --resource-group <...> --template url-template="https://<...>/runtime/webhooks/signalr?code=<...>" --debug
...
az_command_data_logger: command args: signalr upstream update --name {} --resource-group {} --template {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x000002150F3D4180>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x000002150F3D4220>, <function register_cache_arguments..add_cache_arguments at 0x000002150F3D4360>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x000002151149FEC0>]
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x000002150C145E40>, <function CLIQuery.handle_query_parameter at 0x000002150C1739C0>, <function register_ids_argument..parse_ids_arguments at 0x000002150F3D42C0>]
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=SignalRManagementClient
...
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: <...>
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/<...>/resourceGroups/<...>/providers/Microsoft.SignalRService/signalR/<...>?api-version=2023-06-01-preview'
cli.azure.cli.core.sdk.policies: Request method: 'PATCH'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies:     'Content-Type': 'application/json'
cli.azure.cli.core.sdk.policies:     'Content-Length': '306'
cli.azure.cli.core.sdk.policies:     'Accept': 'application/json'
cli.azure.cli.core.sdk.policies:     'x-ms-client-request-id': '6cb48646-9275-11ee-930c-ac1a3db7fa8b'
cli.azure.cli.core.sdk.policies:     'CommandName': 'signalr upstream update'
cli.azure.cli.core.sdk.policies:     'ParameterSetName': '--name --resource-group --template --debug'
cli.azure.cli.core.sdk.policies:     'User-Agent': 'AZURECLI/2.54.0 (MSI) azsdk-python-mgmt-signalr/2.0.0b1 Python/3.11.5 (Windows-10-10.0.22621-SP0)'
cli.azure.cli.core.sdk.policies:     'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: {"location": "", "properties": {"upstream": {"templates": [{"urlTemplate": "https://<...>/runtime/webhooks/signalr?code=<...>"}]}, "publicNetworkAccess": "Enabled", "disableLocalAuth": false, "disableAadAuth": false}}
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "PATCH /subscriptions/<...>/resourceGroups/<...>/providers/Microsoft.SignalRService/signalR/<...>?api-version=2023-06-01-preview HTTP/1.1" 403 2016
cli.azure.cli.core.sdk.policies: Response status: 403
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies:     'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Content-Length': '2016'
cli.azure.cli.core.sdk.policies:     'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies:     'Expires': '-1'
cli.azure.cli.core.sdk.policies:     'x-ms-failure-cause': 'gateway'
cli.azure.cli.core.sdk.policies:     'x-ms-request-id': '<...>'
cli.azure.cli.core.sdk.policies:     'x-ms-correlation-request-id': '<...>'
cli.azure.cli.core.sdk.policies:     'x-ms-routing-request-id': '<...>'
cli.azure.cli.core.sdk.policies:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies:     'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies:     'X-Cache': 'CONFIG_NOCACHE'
cli.azure.cli.core.sdk.policies:     'X-MSEdge-Ref': 'Ref A: <...> Ref B: <...> Ref C: <...>'
cli.azure.cli.core.sdk.policies:     'Date': 'Mon, 04 Dec 2023 07:19:15 GMT'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"error":{"code":"RequestDisallowedByPolicy","target":"<...>","message":"Resource '<...>' was disallowed by policy. Policy identifiers: '[{"policyAssignment":{"name":"Allowed locations","id":"/providers/Microsoft.Management/managementGroups/<...>/providers/Microsoft.Authorization/policyAssignments/<...>"},"policyDefinition":{"name":"Allowed locations","id":"/providers/Microsoft.Authorization/policyDefinitions/<...>"}}]'.","additionalInfo":[{"type":"PolicyViolation","info":{"evaluationDetails":{"evaluatedExpressions":[{"result":"True","expressionKind":"Field","expression":"location","path":"location","expressionValue":"","targetValue":["westeurope","southcentralus","southeastasia","northeurope","northcentralus","eastasia"],"operator":"NotIn"},{"result":"True","expressionKind":"Field","expression":"location","path":"location","expressionValue":"","targetValue":"global","operator":"NotEquals"},{"result":"True","expressionKind":"Field","expression":"type","path":"type","expressionValue":"Microsoft.SignalRService/signalR","targetValue":"Microsoft.AzureActiveDirectory/b2cDirectories","operator":"NotEquals"}]},"policyDefinitionId":"/providers/Microsoft.Authorization/policyDefinitions/<...>","policyDefinitionName":"<...>","policyDefinitionDisplayName":"Allowed locations","policyDefinitionEffect":"deny","policyAssignmentId":"/providers/Microsoft.Management/managementGroups/<...>/providers/Microsoft.Authorization/policyAssignments/<...>","policyAssignmentName":"<...>","policyAssignmentDisplayName":"Allowed locations","policyAssignmentScope":"/providers/Microsoft.Management/managementGroups/<...>","policyAssignmentParameters":{"listOfAllowedLocations":["westeurope","southcentralus","southeastasia","northeurope","northcentralus","eastasia"]},"policyExemptionIds":[]}}]}}
...
Expected behavior
The command should offer the user a possibility to change the default location "" to an allowed location so that the CLI can be used in conjunction with a target resource that has a location policy attached to it.
or
Internally the command should acquire the location of the target resource and attach it to the PATCH request.
Environment Summary
azure-cli                         2.54.0
core                              2.54.0
telemetry                          1.1.0
Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2
Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\HANDAV.azure\cliextensions'
Python (Windows) 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:38:34) [MSC v.1936 64 bit (AMD64)]
Legal docs and information: aka.ms/AzureCliLegal
Additional context
No response

Thank you for opening this issue, we will look into it.

@yonzhan Is there any update on this? We currently have to downgrade to an older verison of azure-cli to keep our production environment alive. Thanks.

@zackliu please help take a look

It's fixed in the latest patch but still need to wait for the release

Tried the fix on 2.64.0 and it works. Thanks


Help Menu Classed As Dev Menu

Need to create a seperate dev menu and make the help menu visible during production
code is in main.js

import devMenuTemplate from "./menu/dev_menu_template";

// Set the application menu
const setApplicationMenu = () => {
const menus = [appMenuTemplate, editMenuTemplate];
if (env.name !== "production") {
menus.push(devMenuTemplate);
}
Menu.setApplicationMenu(Menu.buildFromTemplate(menus));
};

Resolved.


Compile fails on Centos7

Hi,
When running make I get failures. I've also tried compiling the code against the Developer Toolset 7..10.
When using g++ (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
# make
Using gold linker...
Building _release/inotify-info...
---- inotify-info.cpp ----
g++: error: unrecognized command line option ‘-Wimplicit-fallthrough=2’
make: *** [_release/inotify-info.o] Error 1

When using Developer Toolset 9 - g++ (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2)
Using gold linker...
Building _release/inotify-info...
---- inotify-info.cpp ----
inotify-info.cpp:342:67: error: return type ‘struct statx’ is incomplete
  342 | struct statx mystatx( const char *filename, unsigned int mask = 0 )
      |                                                                   ^
inotify-info.cpp: In function ‘void mystatx(const char*, unsigned int)’:
inotify-info.cpp:344:18: error: aggregate ‘statx statxbuf’ has incomplete type and cannot be defined
  344 |     struct statx statxbuf;
      |                  ^~~~~~~~
inotify-info.cpp:345:57: error: ‘AT_STATX_DONT_SYNC’ was not declared in this scope
  345 |     int flags = AT_NO_AUTOMOUNT | AT_SYMLINK_NOFOLLOW | AT_STATX_DONT_SYNC;
      |                                                         ^~~~~~~~~~~~~~~~~~
inotify-info.cpp:347:53: error: invalid use of incomplete type ‘struct statx’
  347 |     if ( statx( 0, filename, flags, mask, &statxbuf ) == -1 )
      |                                                     ^
inotify-info.cpp:342:8: note: forward declaration of ‘struct statx’
  342 | struct statx mystatx( const char *filename, unsigned int mask = 0 )
      |        ^~~~~
inotify-info.cpp: In member function ‘void thread_info_t::add_filename(ino64_t, const char*, const char*, bool)’:
inotify-info.cpp:365:22: error: variable ‘statx statxbuf’ has initializer but incomplete type
  365 |         struct statx statxbuf = mystatx( filename.c_str() );
      |                      ^~~~~~~~
inotify-info.cpp: In function ‘uint32_t find_files_in_inode_set(const std::vector<procinfo_t>&, std::vector<filename_info_t>&)’:
inotify-info.cpp:672:53: error: ‘STATX_INO’ was not declared in this scope; did you mean ‘STA_INS’?
  672 |             thread_info.add_filename( mystatx( "/", STATX_INO ).stx_ino, "/", "", false );
      |                                                     ^~~~~~~~~
      |                                                     STA_INS
make: *** [Makefile:107: _release/inotify-info.o] Error 1

It compiled successfully on Rocky 8.

Do you know what version of the kernel and glibc you're running with? I'm guessing it's old enough that statx just isn't supported? Which I guess we could look at adding a slower non-statx path if really needed. Thanks.

statx() was added to Linux in kernel 4.11; library support was added in glibc 2.28.

@mikesart Thanks for looking, the Centos 7 kernel is still 3.10 and not glibc on 2.17. I'm starting to phasing out my Centos 7 hosts this year as it is getting EOLed next year. Closing this off.

Would it be possible for the utility to detect when the kernel doesn't support the statx or whatever it requires, and print out an appropriate error?
I tried running it on 2.6.32-042stab120.18 . I had to remove -Wimplicit-fallthrough=2 from the Makefile and compile with make CXX=g++-9 on Ubuntu 16. The build was successful, but when running it, it reports zero inotify handles. I am certain I do have inotify handles open, because lsof -p lists them. strace shows events are handled when the watched files change, and the applications using inotify watches react correctly to file changes.
The problem is that inotify-info output is wrong (shows zero handles):
$ _release/inotify-info -vv
0 ignore_dirs:
------------------------------------------------------------------------------
INotify Limits:
  max_queued_events    16,384
  max_user_instances   128
  max_user_watches     524,288
------------------------------------------------------------------------------
       Pid Uid        App         Watches  Instances
------------------------------------------------------------------------------
Total inotify Watches:   0
Total inotify Instances: 0
------------------------------------------------------------------------------

I appreciate that statx is required, but I, and other users in the future will waste time attempting to use this utility on old systems.
Legacy systems are not going away, and it's good form for new-fangled software like this to fail correctly with 'unsupported kernel' or such message. "Total inotify Watches:   0" is a lie.
Thank you.

There is this bit of code in inotify-info.cpp:
// statx() was added to Linux in kernel 4.11; library support was added in glibc 2.28.
#if defined( __linux__ ) && ( ( __GLIBC__ >= 2 && __GLIBC_MINOR__ >= 28 ) || ( __GLIBC__ > 2 ) )

Can you change this to be this:
// statx() was added to Linux in kernel 4.11; library support was added in glibc 2.28.
#if 0

And rebuild the app and see if it works?

@mikesart, I tried the change you suggested and does not work. The behaviour is unchanged. My system used libc6 2.23-0ubuntu11.3, so the conditional evaluates as false with or without this change.
I looked a bit into the situation, and submitted #19, which implements backwards compatibility to kernel 2.6.32. I have not tested earlier kernels. The stat functions are not called at all on old kernels, because individual watches cannot be resolved; the fdinfo files in /proc do not include the necessary device info (the inotify wd:... line)


The windows builds look really suspect

Comment:
I'm wondering why the windows builds were merged in haste without discussions when previous questions were raised to @conda-forge/core regarding their incompatibility with the larger ecosystem.
Lines that download other executable such as this one
https://github.com/conda-forge/openfast-feedstock/pull/46/files#diff-54cf74e113dd3f6d11e092fdb1d888ec82c69bdafbb15cfb6570c83ecad28f33R3

Are not really OK in the conda-forge ecosystem.
I think it would have been important for that build to ping somebody from conda-forge/help-c-cpp  to ensure that things are on the right track.
Please refrain for future merges to the conda-forge repository and lets wait for this particular discussion to be resolved.
Particularly, I am worried about downloading parts that are critical to the windows build in the bld.bat script.

The referenced PR also removes a maintainer from the feedstock without any mentions on "why".

@deslaughter could you please provide some more background here?
Asking as it appears you are the author of PR ( https://github.com/conda-forge/openfast-feedstock/pull/46 ), which is what Mark is asking about above

@jakirkham, @hmaarrfk the OpenFAST developers have struggled for a long time to build this software on Windows using conda-forge. The main difficulty is finding a modern Fortran compiler for Windows as the default one is not compliant with the 2003 standard. Unfortunately, the LLVM Flang compiler, which is used by SciPy, is not yet ready for production use (did not successfully compile OpenFAST because it is missing features). The lack of a Fortran compiler on Windows seems like a known issue (https://github.com/conda-forge/conda-forge.github.io/issues/1044). The best solution that I could find was to use the Intel oneAPI compiler. I agree that downloading the Intel oneAPI compiler in the bld.bat is not the ideal solution, but it will work until Flang is ready to use. Please let me know if there is a better solution available, and I'll be happy to switch to it.
@xhochy the maintainer was removed in the PR because he is no longer active on the project. I should have explained this change.

Thank you for working to address it.
1 year later i find time to reply:
The order in which things is done matters.
You can explain your challenges, then ask if the proposal is OK to merge.
Ultimately, some things are under different licenses and can introduce ABI incompatibility.
When doing things that are strange you can contact @conda-forge/help-c-cpp can get you knowledgeable people in the C/C++/Fortran world.
Please try to communicate challenges in the future.


Updated to Swift 2.0 syntax

Thanks for your pull request, but Xcode 7 is still in Beta, I will update it when it is officially released.


Snippet 'require-module' inserts '-Module' instead of '-Modules'

Before submitting a bug report:

Make sure you are able to repro it on the latest released version
Perform a quick search for existing issues to check if this bug has already been reported

Steps to reproduce
Insert the snippet requires-module, requires-module-version or requires-module-required-version
<# no reproduction code #>

Expected behavior
The snippet inserts a -modules parameter with a s as documented in the about_Requires module section
#Requires -Modules fully-qualified-name

Actual behavior
A -Module is inserted without the final s.
#Requires -Module fully-qualified-name

Note:
The parameter is correctly unrecognized in the tooltip code highlighting:

If an unexpected error was thrown then please report the full error details using e.g. $error[0] | Select-Object *
Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.19041.1023
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.1023
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }

1.19.1

Closed: reported on wrong repo


🌐 Add French translation for docs/fr/docs/virtual-environments.md

Hi ✌
Just a small contribution to the french documentation translation by adding the page ‘docs/fr/docs/virtual-environments.md’.  🇫🇷
Feel free to let me know if you notice any issues, and if any French speakers spot errors, I’ll fix them as soon as possible.
Thanks!

Sorry, I added a page (docs/fr/docs/advanced/sub-applications.md) to the wrong branch… Is it possible to review and have a pull request for 2 pages, or should I abandon this pull request and create one for each page separately?


custom path for svg assets

allow overriding the hard-coded path at: https://github.com/nimiq/iqons/blob/master/src/js/iqons.js#L91

You can now overwrite Iqons.svgPath with the path to the SVG file, as added in 5248be9c7d63e51a45a1e5f2e04f60f5bcf09341.


Switch to new faker

Overview
Update faker dependency
Related to https://github.com/faker-js/faker/issues/542
Screenshots (if applicable)
Documentation

[ ] Updated Typescript types and/or component PropTypes
[ ] Added / modified component docs
[ ] Added / modified Storybook stories

Please run the CI tests so we can see if everything works
I will come back and update to released version when it's out

Hey @Shinigami92 - thanks for your contribution! My apologies - I should've assigned that issue to myself, I pushed out a PR  over the weekend to take care of it (#1421) - it looks like we needed to upgrade to node 14, too.
The changes look pretty similar aside from the direct import of the en locale in your PR - was this intentional or can the import stay as @faker-js/faker?

Oh it's my fault that I didn't checked that there is already an issue / a PR.
I think I will close this PR and we should move on in the other already existing PR


Memory leak when using FixedBufferAllocator

Functions like:
pub fn tagged(allocator: Allocator, tag: u64, value: DataItem) CborError!@This() {
        var di = DataItem{ .tag = Tag{ .number = tag, .content = try allocator.create(DataItem) } };
        di.tag.content.* = value;
        return di;
}

lead to a memory leak when using a FixedBufferAllocator, but std.mem.allocator doesn't detect any leaks.
I can't tell if this is a (FixedBufferAllocator) bug, or related to copying a DataItem with a slice. Copying a slice shouldn't be a problem because it's just a pointer and a length, so the actual data it points to should stay untouched.

Code to reproduce the issue:
test "doom" {
    //var buffer: [256]u8 = undefined;
    //var fba = std.heap.FixedBufferAllocator.init(&buffer);
    //const allocator = fba.allocator(); // <- OutOfMemory
    const allocator = std.testing.allocator; // <- works

    var i: usize = 0;
    while (i < 1024) : (i += 1) {
        var x = try DataItem.text(allocator, "hello, world");
        var di = DataItem{ .tag = Tag{ .number = i, .content = try allocator.create(DataItem) } };
        di.tag.content.* = x;
        defer di.deinit(allocator);
        std.debug.print("i: {d}, c: {s}\n", .{ di.tag.number, di.tag.content.text });
    }
}

Seems like FixedMemoryAllocator doesn't reclaim memory after free (see issue 3049).
Tbh this isn't how a heap is expected to work. Should be documented.


Created Ping pong game

Title and Issue number
Ping Pong ball #2047
Close #2047
Checklist:

[X] I have mentioned the issue number in my Pull Request.
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have created a helpful and easy to understand README.md

https://github.com/pranjay-poddar/Dev-Geeks/assets/133862694/55fde85e-3bcd-4356-8677-3fddcb9f4861

@MohitGupta121 Check out


[bug]:

Is there an existing issue for this?

[X] I have searched the existing issues

OS
Windows
GPU
cuda
VRAM
8gb or 31
What happened?
Hello,
i've tryed to install invoke on my laptop, and after waiting for more than 1 hour i've go an error
ERROR: Wheel 'torch' located at C:\Users\MOI\AppData\Local\Temp\pip-unpack-ekeh6i6s\torch-1.12.1+cu116-cp310-cp310-win_amd64.whl is invalid.
Can you help me to fix it.
Thank you.
Screenshots
Additional context
No response
Contact Details
<EMAIL_ADDRESS>

But did you try to install again? Seems like a simple download failure.
BTW, in most ex-USSR countries Internet Providers randomly block and unblock random addresses. Net based installers like this are almost guaranteed to fail from time to time because of that. Try again later or use a (different) VPN.