Service fabric upgrade timeout failure shows wrong error message

If starting application upgrade fails or times-out, service fabric deploy task outputs wrong error message. It wrongly claims that unregistering application type failed. Also, it tried to unregister application type while upgrade is still in progress
Error log:
2018-03-20T15:49:02.1624650Z ##[section]Starting: Deploy Service Fabric Application
2018-03-20T15:49:02.1631007Z ==============================================================================
2018-03-20T15:49:02.1631433Z Task         : Service Fabric Application Deployment
2018-03-20T15:49:02.1631775Z Description  : Deploy a Service Fabric application to a cluster.
2018-03-20T15:49:02.1632100Z Version      : 1.7.1
2018-03-20T15:49:02.1632406Z Author       : Microsoft Corporation
2018-03-20T15:49:02.1632747Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=820528)
2018-03-20T15:49:02.1633146Z ==============================================================================
2018-03-20T15:49:08.4386091Z Searching for path: D:\a\r1\a\**\projectartifacts\**\PublishProfiles\Cloud.xml
2018-03-20T15:49:08.7031498Z Found path: D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\projectartifacts\src\Services\xyz\PublishProfiles\Cloud.xml
2018-03-20T15:49:08.9570518Z Searching for path: D:\a\r1\a\**\applicationpackage
2018-03-20T15:49:09.0749147Z Found path: D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage
2018-03-20T15:49:09.2471252Z Imported cluster client certificate with thumbprint '********'.
2018-03-20T15:49:09.2768281Z
2018-03-20T15:49:09.2772347Z Thumbprint                                Subject
2018-03-20T15:49:09.2772957Z ----------                                -------
2018-03-20T15:49:09.2773580Z ********  CN=PRIVATE.co.za
2018-03-20T15:49:17.9056327Z Successfully connected to cluster.
2018-03-20T15:49:17.9378376Z Searching for path: D:\a\r1\a
2018-03-20T15:49:18.0744999Z No items were found with search pattern D:\a\r1\a.
2018-03-20T15:49:18.6908613Z ##[warning]The names of some imported commands from the module 'Create-DiffPackage' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2018-03-20T15:49:18.7312943Z Trying to create diff package...
2018-03-20T15:49:19.8220742Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\ApplicationManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\ApplicationManifest.xml in diff package...
2018-03-20T15:49:20.4714898Z Creating diff package for service: Web.ApiPkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:20.5116666Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\Web.ApiPkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\Web.ApiPkg\Code in diff package...
2018-03-20T15:49:28.1601722Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\Web.ApiPkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\Web.ApiPkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:28.1795138Z Creating diff package for service: Services.xyz4Pkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:28.1815985Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\Services.xyz4Pkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\Services.xyz4Pkg\Code in diff package...
2018-03-20T15:49:32.1956118Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\Services.xyz4Pkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\Services.xyz4Pkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:32.2130536Z Creating diff package for service: Services.PersistencePkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:32.2144508Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\Services.PersistencePkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\Services.PersistencePkg\Code in diff package...
2018-03-20T15:49:36.8005672Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\Services.PersistencePkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\Services.PersistencePkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:37.3178177Z Creating diff package for service: PRIVATE.xyz3Pkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:37.3179839Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.xyz3Pkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.xyz3Pkg\Code in diff package...
2018-03-20T15:49:41.7084712Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.xyz3Pkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.xyz3Pkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:41.7121789Z Creating diff package for service: PRIVATE.MakroPkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:41.7203976Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.MakroPkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.MakroPkg\Code in diff package...
2018-03-20T15:49:45.6371084Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.MakroPkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.MakroPkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:45.6534387Z Creating diff package for service: PRIVATE.PickNPayPkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:46.3105587Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.PickNPayPkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.PickNPayPkg\Code in diff package...
2018-03-20T15:49:48.4720910Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.PickNPayPkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.PickNPayPkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:48.4762307Z Creating diff package for service: PRIVATE.xyz2Pkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:48.4777862Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.xyz2Pkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.xyz2Pkg\Code in diff package...
2018-03-20T15:49:52.1711235Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.xyz2Pkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.xyz2Pkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:52.1875962Z Creating diff package for service: PRIVATE.ShopritePkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:52.1889356Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.ShopritePkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.ShopritePkg\Code in diff package...
2018-03-20T15:49:56.2258146Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.ShopritePkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.ShopritePkg\ServiceManifest.xml in diff package...
2018-03-20T15:49:56.2467981Z Creating diff package for service: PRIVATE.xyz1Pkg, clusterServiceManifest.Version: <IP_ADDRESS>80315.2, localServiceManifest.Version: <IP_ADDRESS>80320.2
2018-03-20T15:49:56.2481459Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.xyz1Pkg\Code from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.xyz1Pkg\Code in diff package...
2018-03-20T15:50:02.1179761Z Copying D:\a\r1\a\xyz - Service Fabric\PRIVATE.servicefabric.xyz\applicationpackage\PRIVATE.xyz1Pkg\ServiceManifest.xml from full application package to C:\Users\VssAdministrator\AppData\Local\Temp\DiffPackage\PRIVATE.xyz1Pkg\ServiceManifest.xml in diff package...
2018-03-20T15:50:03.9737494Z Copying application to image store...
2018-03-20T15:52:53.6425444Z Upload to Image Store succeeded
2018-03-20T15:52:53.6482691Z Registering application type...
2018-03-20T16:03:36.3157356Z Register application type succeeded
2018-03-20T16:03:36.3474942Z Start upgrading application...
2018-03-20T16:05:36.8820126Z Unregistering application type 'xyz' and version '<IP_ADDRESS>80320.2' failed
2018-03-20T16:05:36.8871501Z -Message Operation timed out.
2018-03-20T16:05:39.9468720Z ##[error]Application type and version is still in use
2018-03-20T16:05:40.0699389Z ##[section]Finishing: Deploy Service Fabric Application

Looking at the code, this seems to be a regression in showing error message. We will fix this.
However, the failure initiated when Start-ServiceFabricApplicationUpgrade command failed after a timeout of 2 minutes. @oanapl, does starting upgrade also depend on the size of the application type package?

Upgrade depends on many things (check the upgrade parameters first). It may take a long time if there are many UDs and the health parameters are set to high values (eg. health stable duration large). The size of the app can also come into play, since the packages (the changes mostly) need to get on the nodes where the new replicas will be running.

Agree that upgrade can take time, but isn't Start-ServiceFabricApplicationUpgrade command async. We use this command to start upgrade and then track upgrade using Get-ServiceFabricApplicationUpgrade. Why should Start-ServiceFabricApplicationUpgrade not complete even after 2 minutes

My bad, I misunderstood your question, I thought you were asking about upgrade per se, not about start upgrade command.
The start upgrade command does not depend on the size of the package.
Start upgrade could timeout/fail for various reasons, like the message is dropped due to networking, or because CM service primary was moving at the time etc. We can make this more resilient by retrying the command a few times on retryable errors. So each call has a short timeout (default) and add retry for retryble errors until either success or an ubber timeout/max retry count is reached.

@brutaldev , the primary reason that task is failing for you is that starting upgrade itself is timing out. You can specify higher timeout in task UI or in publish profile(if you are using publish profile for upgrade).

do let me know if increasing timeout is fixing the issue.
@oanapl the resiliency you mention would be problematic to do for start upgrade command. We wouldn't want multiple upgrades to get started by each retry. This would work for get commands or for register/unregister commands.


API key not found

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting questions.
[X] I have removed any sensitive information from my code snippets and submission.

App Id
https://us-east-1.console.aws.amazon.com/amplify/home?region=us-east-1#/d3sst100jc4y17
AWS Region
us-east-1
Amplify Hosting feature
Frontend builds
Question
When i run amplify push i got error - "Reason: API key not found: .....my key"
in AWS AppSync newer key.
How i can update aws_appsync_apiKey in amplify

Hi @Ruslan310 this could be due to the API key expiring.
Here's a workaround in a past issue. Can you let me know if this workaround gets you past go: https://github.com/aws-amplify/amplify-cli/issues/2672#issuecomment-557601027

Hey @Ruslan310, Are you still experiencing this issue?

Hey 👋 , This issue is being closed due to inactivity. If you are still experiencing the same problem and need further assistance, please feel free to leave a comment. This will enable us to reopen the issue and provide you with the necessary support.


Remove mentions of persistance

While reading the docs I saw persist_event mentioned but didn't find it anywhere which was confusing, then I saw other mentions of persistence that seem to be just dead code. Didn't find much info on why or when it was removed so this is also a question about what's the deal with persistence? Anyway I thing if the feature comes back we'll always have git to tell us what the code looked like in the meantime is just dead code.

Hi @olanod Thanks for raising this question. Basically, persistence used to be part of Riker 1.0. After Riker 2.0 was released the changes were too significant to make persistence work in time. I didn't want to wait to release Riker 2.0 until persistence was in place. The code was simply commented out with the intention to get persistence working asap. This didn't happen since I was unable to commit time to the project for several months until recently.
We are making some changes to the projects organization with adding people to the team. We will discuss persistence and ask everyone if this is a feature that we should work on.
Thanks for your PR to remove the comments!


Add support for member_role_id

A clear and concise description of what you want to happen.
Add support for member_role_id in https://github.com/gitlabform/gitlabform/blob/main/gitlabform/gitlab/members.py
This will allow for the configuration of custom roles
Thanks.

Hey @adam-moss - Shouldn't this be supported already since the member_role_id is just another attribute of group/project members API.
I didn't check the code yet. Since gitlabform follows the raw parameter passing pattern, in theory it should already be supported. I'm not able to validate it. Would you be able to try?

@adam-moss - did you get a chance to try it out yet?

Thank you for verifying. I just took a quick peak at the code for project membership. Looks like "all the data" from the config is not being passed to the API like I had assumed. What's being passed to the API right now is access_level and expires_at...
https://github.com/gitlabform/gitlabform/blob/f7c592205ac9fe480eacb7340de60987d90541e2/gitlabform/processors/project/members_processor.py#L142-L152
Looking forward to the update. I guess this will only take care of adding an existing "custom role" as member. Maybe we should open a separate feature request to have gitlabform to manage the custom roles?

In my opinion, it makes sense that this is available via python-gitlab library. There's an open issue about it too
https://github.com/python-gitlab/python-gitlab/issues/1357
Of course I can't speak for that library. Hopefully maintainers of the library also agree. Even then not sure if they have the capacity to provide that option.
Don't know if there's another equivalent library. The really last option I think would be to just implement it in gitlabform but my suggestion would be to create a separate project that basically builds the library and then consume it here.

@amimas @adam-moss - I'll look at adding member_role_id into the relevant places on the REST APIs but if we want/need to support creation, then yeah I think adding initial GraphQL support to python-gitlab would probably be the way to go


Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue

Is this package manager not being supported anymore? It would be nice if minimatch could get an update. Thanks

@sirlantis


[cli] use inferred-types when using pull schema

Context
When existing apps pull schema, we're going to write out all attributes as any.
This is a bit frustrating as an upgrade experience, as now developers need to manually update every column with it's actual type.
Potential solution
Well, one thing we do have is inferred types. This PR makes it so if we detect an attribute that has just one inferred type, then we choose that when someone writes pull schema.
Some tradeoffs in doing this
Mainly, it's a bit surprising that we use the inferred type.

The user will have to write push schema, to actually enforce these types. The user may not know this
a. Perhaps it's not too big a deal though, as eventually they will write push schema and see it.
What if the user intentionally wanted to leave an attribute untyped? Right now if they only have strings we'll keep telling him to type it
a. Maybe this isn't too big a deal too, because as the user has heterogeneous data, we will stop doing it.

The two tradeoffs do make me worry, but I think it would lead to a much better upgrade experience if we used inferred types
@dwwoelfel @nezaj @tonsky

I like it! I will merge this into the main strong init PR, and subsume the update as a part of 0.17.0-experimental.4


cannot fit training data to TensorForestEstimator

Currently I am trying to to implement a random forest regression using Tensorflow's TensorForestEstimator. I have successfully done using scikit-learn's RandomForestRegressor and wants to replicate the same result using Tensorflow.
I uploaded the data using pandas and I split the training and test set using scikit-learn's train_test_split. It contains 4 features (all numerical).
>>> X_train.shape
(2711, 4)
>>> y_train.shape
(2711,)

I set the parameters of the for the tree
num_features = int(np.log2(len(clean_data.columns)))

params = ForestHParams(num_classes=1, num_features=num_features,
                       regression=True,num_trees=447, max_nodes=1000)

regressor = TensorForestEstimator(params)

Above i set the features as int(np.log2(len(clean_data.columns))) because i used log2 for the max_features parameter in my original scikit-learn implementation.
However, when tried to fit the training data, i receive and error like such
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    509                 as_ref=input_arg.is_ref,
--> 510                 preferred_dtype=default_dtype)
    511           except TypeError as err:

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in internal_convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, ctx)
   1108     if ret is None:
-> 1109       ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
   1110

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _TensorTensorConversionFunction(t, dtype, name, as_ref)
    945         "Tensor conversion requested dtype %s for Tensor with dtype %s: %r" %
--> 946         (dtype.name, t.dtype.name, str(t)))
    947   return t

ValueError: Tensor conversion requested dtype float32 for Tensor with dtype float64: 'Tensor("concat:0", shape=(?, 4), dtype=float64)'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-18-79323408f7f7> in <module>()
      1 # from tensorflow import cast, float32
      2 # X_train_cast = cast(X_train, float32)
----> 3 regressor.fit(x=X_train, y=y_train)
      4
      5 #regressor.score()

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
    430                 'in a future version' if date is None else ('after %s' % date),
    431                 instructions)
--> 432       return func(*args, **kwargs)
    433     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    434                                        _add_deprecated_arg_notice_to_docstring(

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in fit(self, x, y, input_fn, steps, batch_size, monitors, max_steps)
    506     _verify_input_args(x, y, input_fn, None, batch_size)
    507     if x is not None:
--> 508       SKCompat(self).fit(x, y, batch_size, steps, max_steps, monitors)
    509       return self
    510

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in fit(self, x, y, batch_size, steps, max_steps, monitors)
   1525         steps=steps,
   1526         max_steps=max_steps,
-> 1527         monitors=all_monitors)
   1528     return self
   1529

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
    430                 'in a future version' if date is None else ('after %s' % date),
    431                 instructions)
--> 432       return func(*args, **kwargs)
    433     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    434                                        _add_deprecated_arg_notice_to_docstring(

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in fit(self, x, y, input_fn, steps, batch_size, monitors, max_steps)
    522       hooks.append(basic_session_run_hooks.StopAtStepHook(steps, max_steps))
    523
--> 524     loss = self._train_model(input_fn=input_fn, hooks=hooks)
    525     logging.info('Loss for final step: %s.', loss)
    526     return self

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in _train_model(self, input_fn, hooks)
   1039       self._check_inputs(features, labels)
   1040       training_util._get_or_create_global_step_read()  # pylint: disable=protected-access
-> 1041       model_fn_ops = self._get_train_ops(features, labels)
   1042       ops.add_to_collection(ops.GraphKeys.LOSSES, model_fn_ops.loss)
   1043       all_hooks.extend(hooks)

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in _get_train_ops(self, features, labels)
   1262       `ModelFnOps` object.
   1263     """
-> 1264     return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.TRAIN)
   1265
   1266   def _get_eval_ops(self, features, labels, metrics):

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in _call_model_fn(self, features, labels, mode, metrics, config)
   1225     if 'model_dir' in model_fn_args:
   1226       kwargs['model_dir'] = self.model_dir
-> 1227     model_fn_results = self._model_fn(features, labels, **kwargs)
   1228
   1229     if isinstance(model_fn_results, model_fn_lib.ModelFnOps):

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/tensor_forest/client/random_forest.py in _model_fn(features, labels, mode)
    169
    170     logits, tree_paths, regression_variance = graph_builder.inference_graph(
--> 171         features)
    172
    173     summary.scalar('average_tree_size', graph_builder.average_size())

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/tensor_forest/python/tensor_forest.py in inference_graph(self, input_data, **inference_args)
    512             data_spec,
    513             sparse_features=processed_sparse_features,
--> 514             **inference_args)
    515         probabilities.append(probs)
    516         paths.append(path)

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/tensor_forest/python/tensor_forest.py in inference_graph(self, input_data, data_spec, sparse_features)
    686         sparse_shape,
    687         input_spec=data_spec.SerializeToString(),
--> 688         params=self.params.serialized_params_proto)
    689
    690   def size(self):

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/tensor_forest/python/ops/gen_model_ops.py in tree_predictions_v4(tree_handle, input_data, sparse_input_indices, sparse_input_values, sparse_input_shape, input_spec, params, name)
    467         sparse_input_values=sparse_input_values,
    468         sparse_input_shape=sparse_input_shape, input_spec=input_spec,
--> 469         params=params, name=name)
    470     _result = _op.outputs[:]
    471     _inputs_flat = _op.inputs

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    531             if input_arg.type != types_pb2.DT_INVALID:
    532               raise TypeError("%s expected type of %s." %
--> 533                               (prefix, dtypes.as_dtype(input_arg.type).name))
    534             else:
    535               # Update the maps with the default, if needed.

TypeError: Input 'input_data' of 'TreePredictionsV4' Op has type float64 that does not match expected type of float32.

My assumption was that i have to set the number of features to the number of all the features (i.e. using all features instead of a subset of the features). But i still get the same error as above.
I tried to look at the source code directly but could not really understood where was the issue. A similar issue is being discuss on github here.
Another attempt was converting the input into float32.
regressor.fit(x=X_train.astype("float32"), y=y_train.astype("float32"))
However, i still got the same error as above. Then i tried using tf.cast
X_train_cast = cast(X_train, float32)
y_train_cast = cast(y_train, float32)
regressor.fit(x=X_train_cast, y=y_train_cast)

But i got a different error saying
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-19-89e4fa057afb> in <module>()
      2 X_train_cast = cast(X_train, float32)
      3 y_train_cast = cast(y_train, float32)
----> 4 regressor.fit(x=X_train_cast, y=y_train_cast)
      5
      6 #regressor.score()

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
    430                 'in a future version' if date is None else ('after %s' % date),
    431                 instructions)
--> 432       return func(*args, **kwargs)
    433     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    434                                        _add_deprecated_arg_notice_to_docstring(

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in fit(self, x, y, input_fn, steps, batch_size, monitors, max_steps)
    504     if (steps is not None) and (max_steps is not None):
    505       raise ValueError('Can not provide both steps and max_steps.')
--> 506     _verify_input_args(x, y, input_fn, None, batch_size)
    507     if x is not None:
    508       SKCompat(self).fit(x, y, batch_size, steps, max_steps, monitors)

~/Desktop/88sparses/recommendation/recom/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py in _verify_input_args(x, y, input_fn, feed_fn, batch_size)
    102
    103     if tensor_util.is_tensor(x) or y is not None and tensor_util.is_tensor(y):
--> 104       raise ValueError('Inputs cannot be tensors. Please provide input_fn.')
    105
    106     if feed_fn is not None:

ValueError: Inputs cannot be tensors. Please provide input_fn.

I was wondering if there was something missing in my implementation? Thanks in advance.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

It has been 29 days with no activity and the awaiting response label was assigned. Is this still an issue?

@tensorflowbutler I was able to solve the issue, thank you very much!

@grilhami How did you solve this issue, I am having the exact same issue.
Dataset lookup
X_train,y_train,len(X_train),len(y_train)
(array([[ 3.3042e-01,  2.4995e-01, -6.0874e-01, ...,  3.0400e+02,
          5.0000e+00,  1.0000e+00],
        [ 4.2466e-01,  8.5174e-01,  8.6044e-01, ...,  1.0000e+00,
          7.8000e+01,  1.0000e+00],
        [ 6.1890e-01, -1.1185e+00,  5.8483e-01, ...,  1.4000e+01,
          7.0000e+00,  1.0000e+00],
        ...,
        [ 9.0512e-01,  1.3008e-01,  1.0917e+00, ...,  1.7000e+01,
          2.0000e+00,  1.0000e+00],
        [-1.4751e-01,  5.5556e-01,  1.0764e+00, ...,  1.8000e+01,
          1.3000e+01,  1.0000e+00],
        [-5.0246e-01,  1.2178e+00, -8.0065e-01, ...,  1.0000e+00,
          3.0000e+00,  0.0000e+00]]),
 array([1, 0, 5, ..., 8, 5, 9]),
 510281,
 510281)

Then wrote few lines
tf.contrib.tensor_forest.python.tensor_forest.ForestHParams(num_trees=100, max_nodes=1000,num_classes = len(le.classes_),num_features = 119)
classifier =tf.contrib.tensor_forest.client.random_forest.TensorForestEstimator(params)
classifier.fit(x=X_train, y=y_train)

and got this
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype float64: 'Tensor("concat:0", shape=(?, 119), dtype=float64)'
Later tried this


[Question] Is it possible to have custom input without ref?

Hi! Thank you so much for such a wonderful library!
I noticed that in all the examples, ref is used to access native browser elements. But what if my input is completely custom? For example:

const CustomBoolean = ({initValue, onChange}) => {
   const [value, setValue] = useState(initValue)

   useEffect(() => {
       onChange(value)
   }, [value])

   return (

            setValue(true)}>{value ? 'v': '-'}
            setValue(false)}>{!value ? 'v': '-'}

   )
}

This is just a Boolean input for clarity. But there may be a more complex input, for example, handwritten text recognized by a neural network :)
How can such an input be integrated into the form?
Thank you in advance!

Thanks for creating this issue! In this case just ignore the provided props of the Field component and use setValue instead.

FWIW, if you do not have an input, calling setValue will not update the formStore values. Wasted a bit of time trying to figure that out haha.
To resolve, I'm currently using hidden inputs.
<>
  <CustomInput
    field1={getValue(form, 'field1')}
    field2={getValue(form, 'field2')}
    onField1={v => setValue(form, 'field1', v)}
    onField2={v => setValue(form, 'field2', v)}
    />
    <Form.Field name="field1">{createHiddenInput}</Form.Field>
    <Form.Field name="field2">{createHiddenInput}</Form.Field>
</>

I don't think a hidden field is necessary. Instead, you should wrap your custom field inside our <Field /> component because it manages the lifecycle of the field.

@fabian-hiller, this custom input sets 2 values in the form. I attempted to double wrap the Field component, and that did not appear to work.

Okay. That would be exactly the approach I would take, but I am not currently deep enough into Modular Forms to know directly why it might not work in your case.


replace history when $locationChangeStart is prevented

I'm trying to cancel out all effects of unsuccessfull route in terms of Browser History.
when I call event.preventDefault() in $locationChangeStart listener, I expect the new History token replace the last one.
I mean right now for this :
Route A--->Route B(prevented)--->route C($location.replace called)
I get:
Route A->Route A
in my browser history.
but if history get replaced when I prevent locationChangeStart, I'll get:
Route A
that seems logical.
Any thoughts?

This is duplicate of https://github.com/angular/angular.js/issues/2100 - let's move the discussion in there.


Question about auto_scale

What did auto_scale did in the postprocessing?
For example,
im2 = raw.postprocess(use_camera_wb=True, half_size=True, no_auto_bright=True,no_auto_scale=False, output_bps=16)
im1 = raw.postprocess(use_camera_wb=True, half_size=True, no_auto_bright=True,no_auto_scale=True,  output_bps=16)
The only difference is no_auto_scale. Is there anyway that I can manually scale im1 so that it becomes the same as im2?

I don't know exactly what you're aiming to achieve but rawpy is just a wrapper around the libraw library, so I would advise checking our their docs and asking for help on specific parameters over there.
There are also a bunch of notebooks here: https://github.com/letmaik/rawpy-notebooks/blob/master/README.md  which may help you.


Use this.import() to import JS assets from the inputmask package

Using ember-cli-node-assets is incompatible with Embroider and can also cause issues when used in a nested addon. Ember CLI has provided an import() method for quite some time now, that can take care of a lot of the the same functionality, but is better supported in the edge cases mentioned above.
See also https://github.com/adopted-ember-addons/ember-pikaday/pull/234 for another example of this.

Thanks so much!


Fix/sec dep upgrade

Fixes #100, #102.

:tada: This PR is included in version 2.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[MSAuth] Caused by HTTP 404 Not Found error.

Error: Failed to authenticate Microsoft account

Caused by:
    HTTP 404 Not Found

Very, very likely that I missed something so I'm sorry if something obvious went over my head.
Config below:
[account]
email =<EMAIL_ADDRESS>password = "password"
# Leave the rest empty if you do not have security questions
sq1 = ""
sq2 = ""
sq3 = ""
[config]
offset = 0
auto_offset = false
# Spread (if you are unsure leave it as it is)
spread = 0
microsoft_auth = true
gc_snipe = true
change_skin = false
skin_model = "slim"
skin_filename = "example.png"
# Name queueing (allows you to snipe multiple names sequentially)
# Note: This is an optional feature, leave it as it is if you only want to snipe one name
# Example:
# name_queue = ["Marc", "Dream"]
name_queue = []

Nah, you did nothing wrong. I closed my authentication server to avoid paying so much for hosting costs. Right now in the new version, the authentication code is in the sniper itself to provide more transparency. However it is in beta. But it is pretty much stable right now, just need to update my user guide.
Download the newest version here (if you are using windows get the one with the .exe, the other one is for Linux):
https://github.com/tropicbliss/buckshot/releases/tag/v4.0.0
This version is incompatible with the older config.toml so to save you some time I've made a new config.toml according to the one you provided above (don't worry if it seems shorter, I've made the other options optional):
spread = 0
gc_snipe = true
[[account_entry]]
email = ""
password = ""

Gotcha, everything works perfectly now. Thank you for the straightforward answer.


Userfriendly cheat list

I made this .md document with the cheats that work and those that does not work for making easy to people to identify them. Take a look and consider if is good to publish or not. Thank you for your time and all the good things you all share with us.

Thanks. Can you call the file CHEATS.md?

Thanks. Can you call the file CHEATS.md?

Of course!


[BUG REPORT] - crash when i play

so. when i do a chart i EVERY time get a crash while a play. idk why but its says its null object reference. and i think is happening causse from the spamming notes

i hope some one fix it

also the events gives a null object reference


Updates for MLC

The MLC mlc_chat package was renamed to mlc_llm and features a few other API changes. I'm not familiar enough with plugin to know if this covers all necessary changes for all features, but it is enough for me to download and run models.

@simonw Any chance of merging this? Right now the repo is broken as-is. Thanks!

package name still incorrect:
Worked with *--cpu this way:
llm mlc pip install --pre --force-reinstall \
  mlc-ai-nightly-cpu \
  mlc-llm-nightly-cpu \
  -f https://mlc.ai/wheels


"Error adding entities for domain switch with platform dyson_local" with HA 2022.4.0

After upgrading to HA 2022.4.0, the "night mode" and "continuous monitoring" switches of my Dyson Pure Cool in Home Assistant stopped working, and in the logs I can see the error Error adding entities for domain switch with platform dyson_local and the following traceback:
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 382, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 510, in _async_add_entity
    entry = entity_registry.async_get_or_create(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_registry.py", line 345, in async_get_or_create
    return self.async_update_entity(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_registry.py", line 529, in async_update_entity
    raise ValueError("entity_category must be a valid EntityCategory instance")
ValueError: entity_category must be a valid EntityCategory instance

Additionally I'm now seeing the error Error adding entities for domain sensor with platform dyson_local and the temperature sensor stopped working as well, but I assume this was fixed already in c777eb8ba7d28c468293e59f304e5e1eb3a0abeb.

A new update just has been released, it fixes this issue. Update Dyson-local and try again

Thanks, that was fast! But now running into https://github.com/shenxn/ha-dyson/issues/115. 😅


New Product SSI Authority & Schema Registry

Product name: SSI Credential Issuer
GitHub users to invite: @Phil91 @ntruchsess @evegufy
Validation / approved by: @jjeroch
including new GitHub-team, ArgoCD and Vault
Repository is not needed, work will be done with Tractus-X: https://github.com/eclipse-tractusx/ssi-authority-schema-registry

@evegufy the product-ssi-credential-issuer team was already created via a different issue #525 . Is the product name here wrong?

Hi @stephanbcbauer, the SSI Credential Issuer and SSI Authority & Schema Registry are two different products, so the product name is correct.

Hi @stephanbcbauer I updated the product name in the issue description

@evegufy product-ssi-authority-schema-registry was created

Pending member

Access to ArgoCD and Vault is granted. In case of any questions or errors, please reach out to us.


Watch live replay time is not set when game starts.

BUG 1.
We see Watch({0}) instead of time till the moment that i can see the game.
Start chat with axel, ask him to join and start game, see screenshot.
There should be the time

BUG 2
There is similar behavior:
We see just white (not gray!) Watch instead of time till the moment that i can see the game.

do u have exception in log?

I think I fixed that bug pls reopen if it still exists


AST in detail. Fixes #249

For  #249
Feel free for reject that PR and closed issue, if it is not important for you.

This change is

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

program/program.go, line 245 at r1 (raw file):

		// error is coming from; by looking at the parents of the bad lines.
		goast.Fprint(os.Stdout, p.FileSet, p.File, func(string, reflect.Value) bool {
			return true

I don't understand this change from the original ticket. Is this just to print out more attributes that would be nil?

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

program/program.go, line 245 at r1 (raw file):
Previously, elliotchance (Elliot Chance) wrote…
I don't understand this change from the original ticket. Is this just to print out more attributes that would be nil?

Yes. Feel free to reject.

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

program/program.go, line 245 at r1 (raw file):
Previously, Konstantin8105 (Konstantin) wrote…
Yes. Feel free to reject.

There is already a lot of output when this runs. This would just create more noice if there is no legitimate reason for it to be here.

Comments from Reviewable


URL.appendingPathExtension("") appends a trailing dot

import Foundation
let url = URL(fileURLWithPath: "/tmp/x")
print(url.appendingPathExtension(""))
// Swift 5.10: file:///tmp/x
// Swift 6.0: file:///tmp/x.

This behavior change might be tricky as url.deletingPathExtension().appendingPathExtension(url.pathExtension) won't be equal to url.
Https://github.com/swiftlang/swift-driver/pull/1756 revealed this change. Could you check if this is an intentional change? @jrflat

Not intended, I'll push a fix, thanks for finding this!

@jrflat Thanks! Can we cherry-pick this fix to 6.1 so that we will be able to remove a workaround once we drop 6.0 support?

Yes sounds good I'll cherry-pick to 6.1


Unable to set browser on iOS/iPadOS

NetNewsWire uses an instance of Safari for web-view. It ignores the chosen default browser, nor is there any dialog asking what browser to use with NetNewsWire. As FF and Brave have better privacy protection, I'd like to set the browser NetNewsWire makes use of.

Have you tried to set your default browser to to FireFox or Brave? NetNewsWire does honor that setting when you select the Open in Browser option (for example under the Share menu when viewing the article).
If you are referring to the built in browser, it isn't possible to embed anything except WebKit on iOS. As a matter of fact, all browsers on iOS use WebKit because Apple doesn't allow other rendering engines to be installed. It is better to think of the embedded web view as the NetNewsWire Browser.

@vincode-io thanks for the reply. By using the share menu there is no issue. But that's the share menu. By opening the article in the "built-in browser", a Safari-session will be opened. I knew about the WebKit-only issue, but browsers like Firefox or Brave provide better privacy than Safari, also when using WebKit.

I think this is actually what you are asking for: #3029

@vincode-io yep! Another wording but same need. Can you combine both issues?

What we will due is file it as a duplicate which contains an link back to this issue that we can reference when working on it.
Duplicate of #3029


Deletion of endpoints after platform app

While making some changes for localstack/localstack#6256 in came to light that AWS deletes the endpoints of a deleted platform application.
Changes:

deletion of endpoints when deleting the platform app.
test  that validates that the endpoint doesn't exist after it's app was deleted.

Note: in AWS it takes like 5 seconds to delete the endpoint after the deletion of the app.

Codecov Report

Merging #5247 (df86f00) into master (e004c6d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #5247   +/-   ##
=======================================
  Coverage   95.94%   95.94%
=======================================
  Files         688      688
  Lines       68643    68646    +3
=======================================
+ Hits        65862    65865    +3
  Misses       2781     2781

Flag
Coverage Δ

servertests
36.27% <0.00%> (-0.01%)
:arrow_down:

unittests
95.89% <100.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/sns/models.py
97.22% <100.00%> (+0.01%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e004c6d...df86f00. Read the comment docs.


Addition of 8 Hotel photos

There's a conflict with your request. Please make changes and request again

Rename this file and commit
training_images/not-hotels/images.jpg


Support loading a cube texture from a data buffer

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/15178/merge
Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15178/merge/index.html
To test a playground add it to the URL, for example:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15178/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/15178/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/15178/merge
https://gui.babylonjs.com/?snapshot=refs/pull/15178/merge
https://nme.babylonjs.com/?snapshot=refs/pull/15178/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/15178/merge#BCU1XR#0

WebGL2 visualization test reporter:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15178/merge/testResults/webgl2playwright/index.html

Visualization tests for WebGPU (Experimental)
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15178/merge/testResults/webgpuplaywright/index.html


Fix for double log in fold-changes and for indexing in wilcoxon test

This is a fix for the issue discussed in #517. I implemented it by adding a new parameter to indicate whether the data in adata.X is log-transformed or not, and set it to True as a default, assuming most users will have log-transomed data at this point. This fix is assuming the transformation was done with log1p, as in sc.pp.log1p. The matrix is transformed back to raw counts only for the fold-change calculations - everything else is left the same to maintain the tests in log-space.
I also included a fix for a bug in the indexing that was implemented in the original chunking for the wilcoxon test. This was noted in #469 and the same fix is implemented in PR #471 - thanks for catching that bug!

Hey @a-munoz-rojas and @LuckyMD! Thank you for this!
However, I think this adds a lot of code and computational burden. I'd prefer to make an approximation, which is similar to interchanging log mean with mean log.
x1 and x2 are two arrays storing count values.
log2FC(x1, x2) = log2(mean(x1) / mean(x2)) = log2(mean(x1)) - log2(mean(x2)) = (log(mean(x1)) - log(mean(x1))) / log(2)

In Scanpy, we typically store log1p(x) in the data matrix. For the tests, we have already computed mean(log1p(x)). Hence, the following expression is a one-line edit.
log2FCapprox = log2((expm1(mean(log1p(x1))) / expm1(mean(log1p(x2))))

Obviously, this is an approximation; high values in x are down-weighted by virtue of computing a mean that's weighted with a log1p. The resulting mean(log1p(x)) is therefore rigorously smaller than log1p(mean(x)); if you don't have a lot of outliers, then there will be little difference. If there is a lot of variance, there will be a difference. As one has the same effect both in the numerator and denominator, some of this will cancel out. Hence, one gets an estimator for "effect strength" (after all, this is what we are interested in, that is very similar to log2FC, but more robust to outliers.
I'd prefer this, I have to admit, together with a note in the docs.

I reckon that's a fair consideration. In the end we don't use sc.tl.rank_genes_groups() for complex DE tests that require this amount of detail, but instead for marker gene calculations where it's mainly about ranking genes. It would be interesting to see the error of the approximation though.

Hey @falexwolf, thanks for your note. I apologize for the late reply - I was away at a conference and had little time to work on this.
I agree that this is a cleaner way of doing this and will be more robust to outliers, even if it's at the expense of the approximation. Like @LuckyMD mentioned, this function is generally used for more basic differential testing, so further exploration of differential expression can rely on more complicated downstream analysis. Out of curiosity, I'll also calculate what the error of the approximation is, just to have an idea.
I'm a little limited in bandwidth at the moment, but this is an easy change that I should be able to implement this week. Sorry for the delay - I'll submit the changes soon!

Thank you very much, @a-munoz-rojas!

Hey, sorry for the delayed response. I just pushed the new version with the changes we discussed. I also checked what the error looked like - it's pretty close for most of the cases (I'm attaching a plot with estimated (mean-log) vs "actual" (log-mean) fold changes - as expected, only in the extremes do we start to get a little bit of an underestimation. In my dataset, I got a mean error of about 3%, with the largest error seen in genes with extreme outliers as we expected (the attached violin plot has the largest error, ~70% lower.
test_meanlogerror.pdf
violintest_outlier.pdf

Thank you, this looks good! I'm merging it. :)

@a-munoz-rojas I'm getting the following warning:
/home/user/Code/scanpy/scanpy/tools/_rank_genes_groups.py:203: RuntimeWarning: overflow encountered in expm1
  foldchanges = (np.expm1(means[igroup]) + 1e-9) / (np.expm1(mean_rest) + 1e-9) #add small value to remove 0's

Any ideas?

Here is another one:
/home/user/Code/scanpy/scanpy/tools/_rank_genes_groups.py:203: RuntimeWarning: invalid value encountered in true_divide
  foldchanges = (np.expm1(means[igroup]) + 1e-9) / (np.expm1(mean_rest) + 1e-9) #add small value to remove 0's

Also log_transformed=True parameter is not listed in the documentation.

Wait, where exactly is log_transformed used in the code?

log_transformed disappeared in the last commit here...
Better: pp.log1p should write an attribute to .uns, say simply .uns['log1p'] = True. Depending on that attribute, log2fc is computed by rexponaniating or not.
Also: If trying to call a t-test with non-logarithmized data, a warning should be written.
The overflow and 0 warnings: are you sure you used logarithmized data, Gökcen?

Better: pp.log1p should write an attribute to .uns, say simply .uns['log1p'] = True. Depending on that attribute, log2fc is computed by rexponaniating or not.

Fully agree.

Also: If trying to call a t-test with non-logarithmized data, a warning should be written.

Also agree.

The overflow and 0 warnings: are you sure you used logarithmized data, Gökcen?

Oh true, it wasn't log transformed, true.

Ok, good to read that it wasn't log-transformed!
@Koncopd, could you quickly implement these simple changes? Before continuing to work on the UMAP? These simple changes are for 1.4.1, the UMAP will be for 1.5. Thank you!


Popover not rendering form elements inside itself

Hello everyone. I found this bug after update Bootstrap to the latest version (4.3.1) in my project.
The thing is that popover component is not rendering form elements like ,  or  inside it. This feature was working for the previous version of Bootstrap.
You can find this problem here: https://codepen.io/pcarballeda/pen/pYNzzR

For security purposes, <form> and other interactive elements are no longer whitelisted by default for inclusion in tooltips and popovers. See our sanitizer docs for details on how to update this for yourself. https://getbootstrap.com/docs/4.3/getting-started/javascript/#sanitizer


How can i use insert widget function on wysiwyg mode of a textarea?

Hi,
If enable wysiwyg i cant insert a widget. How can I enable that?
Kind regards
peter

Peter,
It should work just as any other WYSIWYG. Are you having trouble utilizing Magento's editor to insert a widget's code into the contents of the entity's attribute,  or is the trouble with rendering the widget on output to screen?
Sent from my Sprint Samsung Galaxy S® 6.
-------- Original message --------
From: pboethig<EMAIL_ADDRESS>Date: 12/30/2015 6:26 AM (GMT-06:00)
To: "tzyganu/UMC1.9"<EMAIL_ADDRESS>Subject: [UMC1.9] How can i use insert widget function on wysiwyg mode of a textarea? (#112)
Hi,
If enable wysiwyg i cant insert a widget How can I enable that?
Kind regards
peter

Reply to this email directly or view it on GitHubhttps://github.com/tzyganu/UMC1.9/issues/112.

@pboethig Could you please try again to include your attachement(s)? They did not get added into the thread here on GitHub. I'd suggest navigating to this issue directly and posting your reply here.
So am I correct in understanding, that these buttons are missing for you?

yes correct.
I helped me out with using the core methods instead of generated blocks.
teher some small differences.
2016-01-06 15:19 GMT+01:00 Darren Felton<EMAIL_ADDRESS>
@pboethig https://github.com/pboethig Could you please try again to
include your attachement(s)? They did not get added into the thread here on
GitHub. I'd suggest navigating to this issue directly and posting your
reply here.
So am I correct in understanding, that these buttons are missing for you?
[image: pic]
https://cloud.githubusercontent.com/assets/2659558/12144566/37c694ba-b44e-11e5-9c5f-7b72fddba1b8.png
—
Reply to this email directly or view it on GitHub
https://github.com/tzyganu/UMC1.9/issues/112#issuecomment-169333803.

@pboethig This should be working out of the box.
Do you mind attaching this file so I can check out the extension's configuration?
var/modulecreator/package/[Namespace]_[Module].xml.

@tzyganu do you mind closing this due to inactivity please?

Hi Darran,
This is my usecase. I have genereated an entity with some fields. Some of
them are from the type "textarea" and I have chosen the option "WYSIWYG"
Editor during configuration of the new module in UMC Modul Creator.
This worked like a charm and was very handy and easy. It generates an
editform with all the fields. Even the WYSIWYG Editor appears and is
functional. Only the small icon to insert a widget is missing.
I have compared the generated code with the corecodes and found out, that
UMC generates the WYSIWYG Editor in another way than Mage Core does.
Kind Regards
Peter
2016-02-03 17:14 GMT+01:00 Darren Felton<EMAIL_ADDRESS>
@tzyganu https://github.com/tzyganu do you mind closing this due to
inactivity please?
—
Reply to this email directly or view it on GitHub
https://github.com/tzyganu/UMC1.9/issues/112#issuecomment-179318544.


v3.8.1 proposal

v3.8.1 2019-04-12
Due to security concerns this version drops support for Node.js versions < 4.0.0

[3578b2abf0] - deps: explicit limit on supported engines (Refael Ackermann)
[ec8505e15f] - deps: updated tar package version to 4.4.8 (Pobegaylo Maksim) #1713
[6e1e425ffb] - (BREAKING for node < 4) Upgrade to tar v3 (isaacs) #1212
[e6699d13cd] - test: fix addon test for Node.js 12 and V8 7.4 (Richard Lau) #1705
[0c6bf530a0] - lib: use print() for python version detection (GreenAddress) #1534

Checklist

[x] npm install && npm test passes
[X] documentation is changed or added
[X] commit message follows commit guidelines

Description of change

CI: https://ci.nodejs.org/job/nodegyp-test-pull-request/121/

https://github.com/nodejs/node-gyp/commit/fcb0117025 needs the rest of the changes to lib/install.js from https://github.com/nodejs/node-gyp/pull/1212/files#diff-f6618e1cc731d58106a806b7679a7616.
Could you also swap the ordering so that the update to tar 4.4.8 comes last? That way every commit would build in case we'd ever want to bisect.

Rebased over https://github.com/nodejs/node-gyp/pull/1718/commits/6e1e425ffbeca39381de37663431e05af7a00f56
https://ci.nodejs.org/job/nodegyp-test-pull-request/122/

Shouldn't CI jobs for 0.10 and 0.12 be removed from the matrix as support is dropped ?

Shouldn't CI jobs for 0.10 and 0.12 be removed from the matrix as support is dropped ?

Will do, but only after this PR gets some LGTMs.

looks like version should be 4.0.0 according to semver due to breaking changes

Shouldn't this be a semver-minor or semver-major though, since we're dropping support for Node.js 4?

When updating dependencies (in this case node) SemVer does not necessarily require a major release.

What should I do if I update my own dependencies without changing the public API?
That would be considered compatible since it does not affect the public API.
https://semver.org/spec/v2.0.0.html#what-should-i-do-if-i-update-my-own-dependencies-without-changing-the-public-api

Can someone explain what the hold up is here please?

pls merge

Ping @refack
If this includes a breaking change, it should also be a new major version. I believe it should be possible to release a 3.8.1 that just includes the non breaking commits and then to release v4.0.0 right after.

Hi, would the major change be limited to requiring Node > 4?
If so, it would be acceptable to explain in the changelog (if it's visible enough) that projects that are already on a recent Node version can upgrade without problems. At least for Etherpad (https://github.com/ether/etherpad-lite/issues/3598) this would be ok.
A minor version that only solves the vulnerability would clearly even better (if technically possible).

Hi, this issue is blocking CI for many projects, would it be possible to reach a resolution soon?

Well, it looks like dropping support for 0.10 and 0.12 is done to be able to upgrade tar. So dropping this change would also drop the vulnerability fix (making the 3.8.1 release useless). There is no patched version of tar supporting node 0.10 and 0.12.

@richardlau if you release the security fix only as 4.0, this would then require a coordinated effort in the community to upgrade all packages using node-gyp to version 4 ASAP though (as their existing semver constraint would not allow installing the patched version).

Is someone able to make an executive decision on this? I can understand the arguments both ways for minor or major, but it's also worth noting that there already is a coordinated effort by folks trying to figure out how to avoid using this vulnerable version of tar while node-gyp waits to release. A semver major release might be inconvenient, but lots of folks are waiting for that anyway, and npm audit will tell folks what they need to do at that point if they need to make a major version upgrade.

Could you guys prioritize it and make an immediate decision? You know a lot of projects have node-gyp as the dependency, and will absolutely get an error in CI/CD pipeline or cyber security check. I know the license is "provided as is", but I think it's better to make a new release as minor to fix it ASAP, just for the community. Thanks.

Since this appears stuck, I've tagged this on the agenda for this week's TSC meeting.

/cc @nodejs/security-wg

If updating node-tar to a non-vulnerable version breaks support for 0.10/0.12, and if this is the first release to break that support (I have not personally checked that), then this should be a semver-major and 4.0.0.

The tar vulnerability does not look dangerous here -- if one is in position to exploit the tar in node-gyp, they are most likely is position to just ship malicious code in package even without the tar vuln.
So this is not a security problem, strictly speaking.

Raising false positive npm audit warnings is not a good thing, because we don't want to diminish the signal from npm audit in situations when it's actually important.

Yes, releasing that as a semver major would mean that downstream would have to bump the version of this package to get the update. But there is nothing wrong with that. If someone could fix that, it's the node-tar package. I.e. backporting https://github.com/npm/node-tar/commit/b0c58433c22f5e7fe8b1c76373f27e3f81dcd4c8 to previous branches of node-tar would fix this potential vulnerability (and npm audit warning) for both node-gyp and all the other consumers of node-tar.

If doing a semver-major bump of node-gyp due to the min engine requirement bump, it might be worth considering to pick a currently supported/security patched versions of NodeJS  https://github.com/nodejs/Release . With end-of-life of NodeJS 6.x next week, NodeJS 8.x might be the better choice:  "node": ">= 8.0.0"

OK, so I've done the last few releases of node-gyp iirc, and I've been getting emails about this one too. Here's the general problem that I bump up against and why I don't jump straight in to make it happen: releases of node-gyp are kind of critical, we can break very large chunks of the ecosystem if we don't get them right and I have a lot of personal experience with having it break. The test suite is almost non-existent and you just have to browse the issues here to see the diversity of environments that it's used in and the novel ways it can mess up. When I do a release I'm taking some responsibility for that. I'll also very likely end up being the person who has to go do emergency clean-up when the breakage happens. So releases are about far more than just typing npm publish.
It looks like @refack has done a good job of pulling in just the important pieces here so maybe this isn't such a big deal. I'll have a deeper look and maybe push out a 4.0.0 later today.
The divergence on master is getting large so that really needs to be dealt with too. I probably can't be relied on to solve that but I'm also not enthused by the idea of handing out publishing access liberally given what I've said about the state of this codebase and the impact of releases--they're really important to get right and it's really hard to know if they are right!

To @rvagg's comment I can say that breaking 0.10 and 0.12 compatibility going to force some down streams to into major version bumps to quieten questionable relevant npm audit warning. Speaking for node-sass we still maintain BC for 0.10 and 0.12 so we'd need bump a major version to drop old Node support. This is something we have plans to do eventually but we're far from ready.

If someone could fix that, it's the node-tar package. I.e. backporting npm/node-tar@b0c5843 to previous branches of node-tar would fix this potential vulnerability (and npm audit warning) for both node-gyp and all the other consumers of node-tar.

If someone could make this happen, it would be the least disruptive.

@xzyfer Perhaps an issue should be filed to https://github.com/npm/node-tar as the starting point, explaining that the usage of older branches is still high and that updating to<EMAIL_ADDRESS>comes with a major version bump because of dropping support Node.js < 4.0.

@ChALkeR done in https://github.com/npm/node-tar/issues/212

@xzyfer Thanks! I left a comment there and also pinged npm about that.

If there are concerns with releasing this, there is nothing wrong with making it a major release.

v4.0.0 is out, now we have to consider the v3.x branch closed, let's move on to master now which I'm rebasing and will put up a v5.0.0 proposal to see what that might look like

master and v3.x were a bit of a mess so I've done a rebase and force push to master so you may want to update your local copies. The old master is preserved at pre-v4.0.0-master and a v5.0.0 proposal is up at #1723 for discussion so we can clean up the backlog, it's getting a bit out of hand.

Thank you for releasing!!

Is there any plan to cut a node-gyp release that doesn't break support for Node < 4 if a fix for node-tar@2 can be found?

@xzyfer I think it'd be reasonable to cut another v3.x if you can get an old branch of node-tar released that will work with it

What about forking tar, fixing the issue in v2 and to use the fork in node-gyp 3 (ideally we do not need a fork and just a PR and a new release for tar@2)? I know it's just a work around but that's at least something we could do for people who can't update yet.

No need to fork. The node-tar folks have said they're open to accepting a PR.

we're having a hard enough time maintaining node-gyp, I don't think we really need another codebase to manage even if limited in scope

I mentioned this on this thread (https://github.com/nodejs/node-gyp/commit/1456ef27c72cc4e49f9b1d82cbb047cb673156c8) , but I am still having an issue when I update node-gyp.  I run npm install node-gyp and get the correct 4.0.0 package, but then I get an error that I something changed and I need to "Run npm rebuild node-sass to download the binding for your current environment." When I do this, it re-loads the 3.8.x version of node-gyp. Error below from the rebuild of node-sass:
gyp ERR! node -v v9.4.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! This is a bug in node-gyp.
gyp ERR! Try to update node-gyp and file an Issue if it does not help:
gyp ERR! https://github.com/nodejs/node-gyp/issues
Build failed with error code: 7
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>postinstall: node scripts/build.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I have edited some info into the original post so as to make this less confusing.

<EMAIL_ADDRESS>has been released which patches the vulnerability. Would it be possible to cut release a<EMAIL_ADDRESS>with this update so that upstreams aren't forced break old node BC?

NVM looks like node-gyp depended on<EMAIL_ADDRESS>which should mean the issue is resolved.

Github is still reporting the vulnerability as existing because NIST's vulnerability report didn't consider 2.2.2 of tar to be acceptable. So, I contacted NIST directly about 2.2.2 not having the vulnerability, and got the following response:

After review of the CVEs, the information provided, and the configurations we have made the appropriate modifications. Please allow up to 24 hours for these changes to populate on the website and in the data feeds.

So stay tuned, because soon tar 2.2.2 will be recognized as an acceptable, non-vulnerable version, and appropriate versions of node-gyp using<EMAIL_ADDRESS>will no longer be flagged as vulnerable.


Add documentation on how to access containers started via docker-compose in README.md

This is only minor; In the README.md it states that the frontend will be running on port 8080 when started locally, the docker-compose.local.yaml starts it on port 3000 - we should explicitly state this in the README so this does not have to be looked up in the docker-compsoe file itself

I addressed this issue in a new branch and I will create a pull request for it after the merging of frontend-dev branch to avoid breaking git again 😄

Best practice would be:
Merge fix-readme into main > merge main into frontend > merge frontend into main
This way you should also avoid any conflicts


bump RancherOS version and settings

and pin RancherOS version.
Also, the setup is now 2 nodes (1024MB RAM each), by default. There are a couple reasons for this:

Rancher Server leaves very little memory for containers
the hosts page looks way cooler with more than one node ;)

LGTM


Add the introducation to StronglyTypedId

Add the introduction to LessCode.EFCore.StronglyTypedId
As proposed on https://github.com/dotnet/EntityFramework.Docs/issues/4160#issuecomment-1327216363

Thank you, I have resolved all the suggestions.
It shows 'Merging is blocked', what can I do to make it merged?


Shall we extend the abstraction to box like pointers?

While we're at it, it might make sense to create a PointerFamily super-trait, and base everything on it.

The project scope increases
Is it feasible
What does it buy us
What consequences for end user ergonomics.

Started the work on this, in a new branch.
That's the kind of ergonomic issues I was fearing

I see your point about the ergonomics. It can be simplified a little bit better than the error message suggests to <P::Pointer<Self> as RefCounted<Self>>::WeakPointer, but it is not much better.
I also noticed that the StrongMark can be removed without affecting any tests. How was it intended to be used?

I'm not sure it still does this in that context, but the intent was to help the type inference and rust-analyzer... There are still cases where it feels like inference should work but it doesn't.

Oh but you're talking about the StrongMark in RefCounted ! This one is a remnant of the master branch that can be safely be removed.


Can't compile for iOS physical devices

We've added sdk plugin to our project:
flutter_user_sdk: ^0.0.1
After that we can not compile our app on iOS physical devices. There is some issue with on of dependency:
Could not build the precompiled application for the device.
Error (Xcode): '/Users/marcinmakurat/furgonetka/application-flutter/build/ios/Debug-iphoneos/awesome_notifications/awesome_notifications.framework/awesome_notifications(awesome_notifications-dummy.o)' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target. file '/Users/marcinmakurat/furgonetka/application-flutter/build/ios/Debug-iphoneos/awesome_notifications/awesome_notifications.framework/awesome_notifications' for architecture arm64

We didn't notice the above problem on iOS simulators and android devices (also physical). We use Fluter 2.10.5.

Hey @bigBoy-mm,
Thank you for your report. We're analyzing your issue. The update should be released soon.
I will keep you updated

Hi,
Flutter introduced version 3.0.0 and some packages adjusted API.
That probably caused problem mentioned above.
Please import this ref if You use Flutter 2.0.0-2.10.5
flutter_user_sdk:
git:
url: https://github.com/UserEngage/Flutter-SDK.git
ref: c4d301576ad1759e777244d573705a753d9c47fe
Also check our docs to properly integrate package:
https://user.com/en/mobile-sdk/installation-flutter/
You also can try to clean build folder with 'flutter clean' before running project.


Can I use this gem with backbone-rails gem ?

I just add two gems in my Gemfile and create one template to test and.. it doesn't work
Gemfile
ExecJS::ProgramError in Main#index
Showing C:/preject/app/views/layouts/application.html.haml where line #6 raised:
TypeError: Object doesn't support this property or method
(in C:/preject/app/assets/javascripts/backbone/templates/profile/index.jst.hamlc)
In index.jst.hamlc just:
.qwe test123

I've used both gems in a project without problems. What JS runtime are you using? I suggest to test with the Ruby racer by uncomment gem 'therubyracer' or install NodeJS.

I tried to install therubyracer gem: error
I think it is because my platform is windows.

According to the error message it needs Python for building libv8. The other option is to use install NodeJS, they have a windows installer. You can verify the ExecJS runtime in the console:
$ irb
> require 'execjs'
> ExecJS.runtime
=> #<ExecJS::RubyRacerRuntime:0x007fae2b0ba6b8>

Ok, I installed NodeJS
irb(main):001:0> require 'execjs'
=> true
irb(main):002:0> ExecJS.runtime
=> #<ExecJS::ExternalRuntime:0x2386db8 @name="JScript", @command="cscript //E:jscript //Nologo //U", @runner_path="C:/Ruby193/lib/ruby/gems/1.9.1/gems/execjs-1.3.0/lib/execjs/support/jscript_runner.js", @test_args=nil, @test_match=nil, @encoding="UTF-16LE", @binary="cscript //E:jscript //Nologo //U">
irb(main):003:0> ExecJS.runtime
But, error is still exist.

Somehow the output here is missing the info, but I got it in the notification email:
irb(main):001:0> require 'execjs'
=> true
irb(main):002:0> ExecJS.runtime
=> #<ExecJS::ExternalRuntime:0x2386db8 @name="JScript", @command="cscript //E:jscript //Nologo //U", @runner_path="C:/Ruby193/lib/ruby/gems/1.9.1/gems/execjs-1.3.0/lib/execjs/support/jscript_runner.js", @test_args=nil, @test_match=nil, @encoding="UTF-16LE", @binary="cscript //E:jscript //Nologo //U">
irb(main):003:0> ExecJS.runtime

You still have JScript as engine selected. Make sure Node is in your path, so it can be detected.

Node is in your path
How to realize it? I'am newbie =)

%PATH% is a list of directories where Windows will search for executables, so you have to add the location where the node command can be found.

ok

But there already is a reference to nodeJS: Files \ Git \ cmd; C: \ Users \ roma \ AppData \ Roaming \ npm; C: \ Program Files \ nodejs \

How do you start your server? From the command line or an editor?

From the command line

Make sure the path is right in Ruby and the node runtime is available:
$ pry
[1] pry(main)> ENV['PATH']
=> "/usr/local/bin:/usr/local/share/npm/bin:/Users/michi/node_modules/.bin:/Users/michi/.bin:/Users/michi/.rvm/gems/ruby-1.9.3-p125/bin:/Users/michi/.rvm/gems/ruby-1.9.3-p125@global/bin:/Users/michi/.rvm/rubies/ruby-1.9.3-p125/bin:/Users/michi/.rvm/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/share/npm/bin"
[2] pry(main)> require 'execjs'
=> true
[3] pry(main)> ExecJS::Runtimes::Node.available?
=> true

According to ExecJS, JScript is the least desired runtime, guess they have good reason to do so.
If you still have problem getting the node runner recognized, have a look at the relevant lines for locating it in ExecJS.

ExecJS::Runtimes::Node.available? return true
but libv8 gem is not established
appears not just me: 1 2 3

You're mixing the runtimes. libv8 is a dependency for the Ruby racer, but when you've NodeJS as runtime, you don't need the Ruby racer, so you can uncomment gem "therubyracer" in your Gemfile.

uncomment gem "therubyracer" ? it's uncommented

Have you run bundle after uncommenting? You can see which Gem has it as dependency within Gemfile.lock.

Have you run bundle after uncommenting? yes

It still tries to install the Ruby Racer.  How did you name your template?

Template why it is transformed into index.hamlc.js

You may have a unicorn in between your browser and the server, that dislocates the template with a space-time bending. It's just a guess, since I don't have more information.

Since I don't have the patience anymore to ask thousand of questions to get all the needed informations that let me help you, I suggest that you put the project on GitHub and send me the link, so I can have a quick look.

ok, i wrote you private message

Thanks, I cloned your repo and found the problem: You had the templates index.jst.ejs and index.jst.hamlc in the same folder. This is not possible, since both files will be compiled to index.js. I renamed index.jst.hamlc simply to index2.jst.hamlc and the template was compiled fine and server through the asset pipeline.


Label sets of faces (2D)

This PR adds support for storing the contours / segments defining the boundaries.
Closes #136
Todo

[x] #146 -> rebase / merge
[ ] Write unit tests
[ ] Use MeshContainer as output from triangulate
[ ] Use MeshContainer to store edges/edge markers
[ ] Add plotting function from notebook (triangles/lines) as method on MeshContainer

Could do something like this to implement the plot on MeshContainer directly:
fig, (ax1, ax2) = plt.subplots(ncols=2)
if set(cell_data.keys()) == set('line', 'triangle'):
    self.plot_mpl('line', ax=ax1)
    self.plot_mpl('triangle', ax=ax2)
plt.show()


Manga Review data often missing due to bad parsing

Current Behavior
https://api.jikan.moe/v4/manga/62795/reviews returns nothing, but
https://myanimelist.net/manga/62795/Tomodachi_Game?q=Tomadachi Game&cat=manga has reviews
I noticed there were existing issues opened around the anime endpoint and 500 errors, but in this case, I'm getting nothing back (empty result), however there should be data present.
Expected Behavior
The reviews on the website should be reproduced
Steps To Reproduce

Hit https://api.jikan.moe/v4/manga/62795/reviews
See no review
Open https://myanimelist.net/manga/62795/Tomodachi_Game?q=Tomadachi Game&cat=manga
Validate there are multiple review

This can be repeated for other manga, like Killing Bites
Environment
Used public API
Firefox/Chrome or via JikanDotNet

Anything else?
This has been validated to be a parsing issue.
https://github.com/jikan-me/jikan-rest/issues/407

Not a bug, responded here: https://github.com/jikan-me/jikan-rest/issues/407#issuecomment-1627759277


performance on amd 7950x ...

Hello,
I tried benchmark on 7950x cpu and performance is in some tests up to 2.3x faster but in other tests much slower (like 0.3x) compared to classical sorting. Is amd implementation of avx512 not so powerful (and your code is not suitable for zen4) or is it something else ?
Thanks,
Jan

Good question! You can use godbilt to check, but try march=native when compiling.

 You can try clang.

Zen 4 based CPUs perform poorly because AMD's implementation of compressstoreu is extremely inefficient (~142 cycles latency / throughput of 42 according to uops.info). It is even worse than an 'emulated' version.
I've also run the benchmark on my 7700X CPU and also get extremely poor results for Zen 4.
Thankfully, replacing calls to compressstoreu with their emulated versions significantly improves the results.
I'm going to push the fix on a fork and will make a PR.

Thanks for the explanation. It is good that the instruction can be emulated and results can be similar to intel speedup.

https://github.com/natmaurice/x86-simd-sort/commit/41d03b2d8f3b62a2ee6a3a97a8da7f193a407026

The answer for slow performance of AVX512 version of x86-simd-sort on Zen 4 is most probably explained in AMD manuals which could be found at: https://www.amd.com/en/search/documentation/hub.html#q=software optimization guide for the amd microarchitecture&f-amd_document_type=Software Optimization Guides
Software Optimization Guide for the AMD Zen4 Microarchitecture has following remark in "2.11.2 Code recommendations" chapter:

Avoid the memory destination form of COMPRESS instructions.  These forms are implemented using microcode and achieve a lower store bandwidth than their register destination forms which use fastpath macro ops.

Software Optimization Guide for the AMD Zen5 Microarchitecture doesn't have any remark about COMPRESS instructions.
Could you add some code that disables the AVX512 version on Zen4, but keeps it enabled on Zen5 and future Zen architectures?

https://web.archive.org/web/20230319232625/https://github.com/natmaurice/x86-simd-sort/commit/41d03b2d8f3b62a2ee6a3a97a8da7f193a407026
OOPS

@r-devulap I'm thinking more about adding Zen 4 AVX-512 exclusion to runtime dispatch, i.e. probably here: https://github.com/intel/x86-simd-sort/blob/59e298d8c9d1bee2cded744b9adbe31107ee220c/lib/x86simdsort.cpp
In short the idea is: if CPU has AVX-512, but is Zen 4, then use AVX2. Otherwise use AVX-512 on Zen 5 and future ones. That would have less negative performance consequences than solutions like https://github.com/openjdk/jdk/pull/16124

Restriction of the AVX512 sort acceleration to only Intel CPUs. A performance regression (due to micro-architectural differences) was reported for AMD Zen4 CPUs in the comments section of PR.

But AVX512 is faster?? Why would you use AVX2?

@tarsa I am fine with that approach. We could start with disabling avx512 on zen4 and eventually get to enabling it with a SW_VCOMPRESS enabled. I am happy to review to a pull request if someone can work on it and provide benchmarks. I am on travel for the next 6 weeks and so I might not get to it immediately.


[Feature request] Allow custom downloader in scripts

It would be nice if one could set a custom command for downloading before invoking the scripts. Since some of the downloads are pretty large, having a downloader that supports multiple connections such as aria2c or axel can make the difference between waiting 5 minutes, or waiting 3 seconds in travis.
Something like
export QTCI_DOWNLOAD='axel -q -c -n 16'
./install-qt 5.10.1

Internally, QTCI_DOWNLOAD could be set to a default value of 'wget -c -nv' if not defined previously.

done

Thank you! The PR only takes care of the Qt download, and not of any others such as the Android SDK, so I'll let this issue open for now.

@mrgreywater
as far as i see, you can pass QT_VERSION to "install-qt" script  as "android-5.5.1" instead of "5.5.1" and it will works. in my opinion we used to have one scrept to "Rule them all" ))

There are quite a few more downloads that use wget and not the custom downloader, see
https://github.com/benlau/qtci/search?q=wget&unscoped_q=wget

as you can see at that search resoult, almost all of cases where covered by my one script. only 2 cases where not covered:

Downloading Android NDK...
Downloading version for MAC

so, if you wanna, you can submit patch to that cases ...


installing dropout_layer_norm

I am trying to install drop_layer_norm to use with fused_dropout_add_ln but the pip installation is taking over an hour to install (and not yet done but also no errors). Is this normal?

It takes about 5-6 minutes to install if ninja is available to parallelize the build (ninja should be come with Pytorch but you can check ninja --version and use htop to check if many CPU cores are being used).

Thanks for the very quick response. I do have ninja installed so not sure where the issue is. The following are the specs.
ninja --version
1.11.1.git.kitware.jobserver-1
gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:18:20_PST_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
The installation of main package works fine though.

Are multiple CPU cores being used to compile?
You can check with htop.

yup. 24 cores all seem to be fully occupied.
I do NOT appear to have the same issue with all the packages in csrc directory -- e.g., rotary, or fused_dense_lib

When I run with pip install -v, It also appears to be stuck at processing ln_fwd_xxx.cu and ln_bws_xxx.cu files --its still progressing, but very slowly.

dropout_layer_norm uses a lot of templating so compilation can be slow, but it takes about 5-6 mins in my case (and the main package takes 4-5 mins for similar reasons).
I'm not sure why it would take much longer to compile.
You can comment out these 2 lines if you're only using Ampere and that should reduce compilation time by 2x.

Thanks, Ill try this.. One of the warnings I get is this, but a little beyond me to know if this is an issue. though.

Even with the modification, it is running for over 1hr now. Btw, do you have a docker with all the flash-attn packages compiled?

We have a dockerfile here.
dropout_layer_norm isn't that important, you can disable it if you're using our training script with model.config.fused_dropout_add_ln=False.

I'm facing the same issue. It stuck while installing dropout_layer_norm.

I figured that it is faster if we use 11.8 cuda but gets stuck on lower versions. Not sure why though. Another hack is to go through the list of .cu files in the setup and only install the fwd/bwd libs for the dimensions you care about.

dropout_layer_norm isn't that important

@tridao out of curiosity, could you share some insight on the performance impact of fused_dropout_add_ln?

@shijie-wu I've seen performance improvement on the order of 1-4% with fused_dropout_add_ln, depending on model size.

is there any rule of thumb you could share on the relationship between improvement and model size?

Larger models will get less improvement, since layer norm (and dropout & residual) will take less time relative to matrix multiply in the MLP and attention.
You can profile your model to see what fraction of time is spent on layer norm etc.

on unrelated note, what is the typical speed up with fused_dense_lib in your experience?

I don't remember exactly, maybe on the order of 2-7%. Larger models will get less improvement, for the same reason above.
You can profile your model to get the exact numbers.

thanks for sharing!

To add another data point I am seeing exactly the same issue until it eventually hangs (?) at the following point:
[46/57] /apps/cuda/RHEL8/cuda_11.6.0/bin/nvcc  -I/home/bjorn/flash-attention/csrc/layer_norm -I/home/bjorn/Megatron-LM/.env/lib/python3.8/site-packages/torch/include -I/home/bjorn/Megatron-LM/.env/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/bjorn/Megatron-LM/.env/lib/python3.8/site-packages/torch/include/TH -I/home/bjorn/Megatron-LM/.env/lib/python3.8/site-packages/torch/include/THC -I/apps/cuda/RHEL8/cuda_11.6.0/include -I/home/bjorn/Megatron-LM/.env/include -c -c /home/bjorn/flash-attention/csrc/layer_norm/ln_fwd_2560.cu -o /home/bjorn/flash-attention/csrc/layer_norm/build/temp.linux-x86_64-cpython-38/ln_fwd_2560.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --expt-relaxed-constexpr --expt-extended-lambda --use_fast_math -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=dropout_layer_norm -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14

I've seen reports (#268) saying using the Nvidia using the Pytorch docker container makes compiling much faster. That's that docker I use.

I can confirm I was seeing the same result, stuck on the exact same line [46/57] /apps/cuda/RHEL8/cuda_11.6.0/bin/nvcc for hours with no end in sight.  This was when I was testing using a vanilla Runpod Pytorch v1 container, I could do everything else except I'd always get stuck on that line.  This would still happen even if I installed ninja (couldn't get past flash-attn install without ninja, or it would take so long I never let it finish).  This was using 128vCPUs, and I also noticed my usage never exceeded 20%, and it never seemed to finish (waited 3 hours on one run).
After switching to the container mentioned nvcr.io/nvidia/pytorch:22.12-py3, not only did the setup go faster, but that line it was hanging on 46/57 took no longer than any other steps.  I also noted my CPU usage stayed around 100% during the build of layer_norm.
Not sure if this is helpful for anyone but thought I'd share.


helm search hub should optionally return the repo URL as part of the results

It would be nice if helm search hub could give the URL of the repo (not necessarily by default, but as an optional column/JSON key), so as to avoid a roundtrip to the Hub description page for the actual repo URL if I can tell from the search listings which chart repo I want.

This is a great suggestion, @omkensey. I think adding another field to the table output displaying the repository URL would be a welcome change.
A little bit of context here: The idea with helm search hub returning that URL was that the chart repository maintainers were looking to add support to use that URL as input to helm install, like so:
><> helm search hub wordpress --max-col-width 0 | tail -n 1
https://hub.helm.sh/charts/bitnami/wordpress           	9.1.1        	5.3.2      	Web publishing platform for building blogs and websites.

then:
><> helm install wordpress https://hub.helm.sh/charts/bitnami/wordpress

cc'ing @mattfarina as he may have more context on where we are with that particular feature.

A little bit of context here: ...
Thank you. I did not see this in the doc. Now I found it in content/docs/intro/using_helm.md since I knew what to look for.

I like to work on this

I like the idea of adding the column. I would also like to see ><> helm install wordpress https://hub.helm.sh/charts/bitnami/wordpress work. This might be something to add into the Artifact Hub as well.


Edits based on Sam Wenker's and John Simmons' review

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Docs/Overview/client-architectures.md
:white_check_mark:Succeeded

Docs/Overview/domain-server.md
:white_check_mark:Succeeded

Docs/Overview/glossary-and-abbreviations.md
:white_check_mark:Succeeded

Docs/Overview/license-acquisition.md
:white_check_mark:Succeeded

Docs/Overview/policies-best-practices.md
:white_check_mark:Succeeded

Docs/Overview/scenario-rental-content.md
:white_check_mark:Succeeded

Docs/images/image26_13.jpg
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Terminal stuck on download

I'm trying to setup libpostal locally on my mac.
After running make on terminal, it's stuck on
Downloading part 11: filename=[Root-Path]/libpostal/libpostal/parser.tar.gz.11, offset=671088640, max=752483239
I have checked the internet, it's working fine.
The setup terminates with the following error code:

make[2]: *** [all-local] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

same problem. download  from url  stucks after 98%. tried multiple times. Any idea


Fix and simplify directories handling

Hi, this change causes it to delete all volumes, in use or not, dir is now the full path and because of that never in allvolumes.

Oops, sorry about that.
Here is a new version.


Fixed #32723 -- Add a GitHub action to run the Sphinx linkcheck builder

ticket-32723
I've create a GitHub action to run the Sphinx linkcheck builder and download the output in an artifact as suggested @felixxm.
Note: This PR is linked to #14325.

Thanks @sabderemane, but that ticket is in Someday/Maybe state as we're awaiting a patch in Sphinx to allow certain "redirected" links to be treated as "working". This will allow us to set up the action to fail if it encounters either "broken" or "redirected" links, and pass I otherwise making it easy to determine whether corrections need to be made.


Remove version: Mythicsoft.AgentRansack version 9.2.3405.1

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

For moderators
This account is automated by Github Actions and the source code was created by CoolPlayLin. If you have any questions about any pull request, don't hesitate to ping Exorcism0666.

[!warning]
With the new version of Komac, numerous Pull Requests may contain bugs or unwanted elements. If such issues arise, please tag russellbanks so that these bugs can be addressed in future updates.

[!important]
Please carefully review these Pull Request before merging. If it is a Pull Request for removing incorrect content and the URLs are issue, free upon manual checking, please close this Pull Request directly. (It is best to inform Exorcism0666 of the closure.)

良い一日をお過ごしください！

URL: https://download.mythicsoft.com/flp/3405/agentransack_3405.exe
Response Error: The remote server returned an error: (403) Forbidden.
(Automated message - build 898)


Now also support changing the option maxTouchDrift

Currently withint the touchEnd event, SwiftClick calculates with an offset of 30px to determine whether the user tried to click or drag. 30px is quite a large offset, if we compare this to fastClick for instance (which uses 10px).
A public API was needed to be able to configure this option.

@DennisJamin you can already set this with SwiftClick.options.maxTouchDrift

In fact, perhaps you are right. It now make sense to remove the whole options object and replace it with "private" variables, since each option is already exposed via the public API.
I'll look into doing this sometime soon.

@DennisJamin before I merge this, can you please also add a check for the maxTouchDrift parameter to ensure it is greater than something like 5? Otherwise, if 0 is passed then no clicks will work.


Disable XNNPACK for s390x machines

This PR disable XNNPACK related build targets and function definitions on s390x because XNNPACK is not supported on s390x: https://github.com/google/XNNPACK#supported-architectures
Without this patch, a lot of Lite targets that are not necessarily depending on XNNPACK dependencies will fail to build on s390x, and this patch will allow them to build successfully

@lintian06/@terryheo Can you please review this PR ? Thanks!

The same applies to PPC