Don't get deterministic results with NGC 20.09

Prerequisites
I am using latest TensorFlow NGC container nvcr.io/nvidia/tensorflow:20.09-tf2-py3
Issue
When running my Deep Learning model on 1 GPU (Tesla T4) within the NGC container 20.09, I don't get reproductible results between consecutive runs (different predictions) although all the seeds are set correctly. Running the same code on CPU gives reproductible results over consecutive runs.
On a small dataset (138k samples), I managed to get reproductible results within the NGC by setting the default float type to float64 (instead of float32).
But with a bigger dataset (2.6m samples), after 1million of samples, the predictions start to differ over different runs.
The more I increase the dataset size, the bigger is the difference between predictions of different consecutive runs.
My datasets are tfrecord files of hashed values, serialized by batch of 10,000 samples.
Here are the small_dataset (size 13M):
small_dataset
and the big_dataset (size 251M) - can you send me your email to share it with you via Google Drive to be able to reproduce the issue?
You need to unzip them before running the script bellow.
Command lines:
sudo docker run --gpus all --shm-size=1g --ulimit memlock=-1 -it --rm -v /home:/workspace/home nvcr.io/nvidia/tensorflow:20.09-tf2-py3

CUDA_VISIBLE_DEVICES="0" TF_DETERMINISTIC_OPS="1" TF_CPP_MIN_LOG_LEVEL="3" python code.py data_file prediction_file

Script
import tensorflow as tf
import sys
import string
import os
from tensorflow.python.keras.initializers import RandomNormal, GlorotUniform, Zeros, glorot_normal
from tensorflow.python.keras.regularizers import l2
from tensorflow.compat.v1.keras.layers import Layer, Embedding, Input, Dense, Flatten, Add
from tensorflow.python.keras import backend as K
import numpy as np
import logging
tf.get_logger().setLevel(logging.ERROR)
K.set_floatx('float64')

prediction_output_name = "pred"

SEED = 1024
os.environ['PYTHONHASHSEED'] = str(SEED)
np.random.seed(SEED)
tf.compat.v1.set_random_seed(SEED)

class Linear(Layer):

    def call(self, x):
        return tf.reduce_sum(x, axis=[1])

    def compute_output_shape(self):
        return None, 1

class FM(Layer):

    def __init__(self, **kwargs):

        super(FM, self).__init__(**kwargs)

    def build(self, input_shape):
        if len(input_shape) != 3:
            raise ValueError("Unexpected inputs dimensions % d,\
                              expect to be 3 dimensions" % (len(input_shape)))
        super(FM, self).build(input_shape)

    def call(self, inputs):
        if K.ndim(inputs) != 3:
            raise ValueError(
                "Unexpected inputs dimensions %d, expect to be 3 dimensions"
                % (K.ndim(inputs)))

        concated_embeds_value = inputs
        square_of_sum = tf.square(tf.compat.v1.reduce_sum(
            concated_embeds_value, axis=1, keep_dims=True))
        sum_of_square = tf.compat.v1.reduce_sum(
            concated_embeds_value * concated_embeds_value, axis=1, keep_dims=True)
        cross_term = square_of_sum - sum_of_square
        cross_term = 0.5 * tf.compat.v1.reduce_sum(cross_term, axis=2, keep_dims=False)

        return cross_term

    def compute_output_shape(self):
        return None, 1

class DNN(Layer):

    def __init__(self, hidden_units, activation='relu', l2_reg=0, dropout_rate=0, use_bn=False, seed=1024, **kwargs):
        self.hidden_units = hidden_units
        self.activation = activation
        self.dropout_rate = dropout_rate
        self.seed = seed
        self.l2_reg = l2_reg
        self.use_bn = use_bn
        super(DNN, self).__init__(**kwargs)

    def build(self, input_shape):
        input_size = input_shape[-1]
        hidden_units = [int(input_size)] + list(self.hidden_units)
        self.kernels = [self.add_weight(name='kernel' + str(i),
                                        shape=(
                                            hidden_units[i], hidden_units[i + 1]),
                                        initializer=glorot_normal(
                                            seed=self.seed),
                                        # initializer=Constant(0.4),
                                        regularizer=l2(self.l2_reg),
                                        trainable=True) for i in range(len(self.hidden_units))]
        self.bias = [self.add_weight(name='bias' + str(i),
                                     shape=(self.hidden_units[i],),
                                     initializer=Zeros(),
                                     trainable=True) for i in range(len(self.hidden_units))]
        if self.use_bn:
            self.bn_layers = [tf.keras.layers.BatchNormalization() for _ in range(len(self.hidden_units))]

        self.dropout_layers = [tf.keras.layers.Dropout(self.dropout_rate, seed=self.seed + i) for i in
                               range(len(self.hidden_units))]

        self.activation_layers = [tf.keras.layers.Activation(self.activation) for _ in range(len(self.hidden_units))]
        super(DNN, self).build(input_shape)

    def call(self, inputs, training=None):

        deep_input = inputs

        for i in range(len(self.hidden_units)):
            fc = tf.math.add(tf.tensordot(
                deep_input, self.kernels[i], axes=(-1, 0)), self.bias[i])

            if self.use_bn:
                fc = self.bn_layers[i](fc, training=training)

            fc = self.activation_layers[i](fc)

            fc = self.dropout_layers[i](fc, training=training)
            deep_input = fc

        return deep_input

    def compute_output_shape(self, input_shape):
        if len(self.hidden_units) > 0:
            shape = input_shape[:-1] + (self.hidden_units[-1],)
        else:
            shape = input_shape

        return tuple(shape)

    def get_config(self, ):
        config = {'activation': self.activation, 'hidden_units': self.hidden_units,
                  'l2_reg': self.l2_reg, 'use_bn': self.use_bn, 'dropout_rate': self.dropout_rate, 'seed': self.seed}
        base_config = super(DNN, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

class PredictionLayer(Layer):

    def __init__(self, task='binary', use_bias=True, **kwargs):
        if task not in ["binary", "multiclass", "regression"]:
            raise ValueError("task must be binary,multiclass or regression")
        self.task = task
        self.use_bias = use_bias
        super(PredictionLayer, self).__init__(**kwargs)

    def build(self, input_shape):

        if self.use_bias:
            self.global_bias = self.add_weight(
                shape=(1,), initializer=Zeros(), name="global_bias")

        super(PredictionLayer, self).build(input_shape)

    def call(self, inputs):
        x = inputs
        if self.use_bias:
            x = tf.nn.bias_add(x, self.global_bias, data_format='NHWC')
        if self.task == "binary":
            x = tf.sigmoid(x)

        output = tf.cast(tf.reshape(x, (-1, 1)), tf.float32)

        return output

    def compute_output_shape(self):
        return None, 1

    def get_config(self, ):
        config = {'task': self.task, 'use_bias': self.use_bias}
        base_config = super(PredictionLayer, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

def input_pipeline(dataset_file, serialized_features, epochs):
    dataset = create_dataset(dataset_file, serialized_features, epochs)
    dataset.prefetch(1)
    iterator = tf.compat.v1.data.make_one_shot_iterator(dataset)
    training_init_op = iterator.make_initializer(dataset)
    return iterator, training_init_op

def create_dataset(dataset_file, serialized_features, epochs):
    decode_func = make_decoder(serialized_features)
    return tf.data.TFRecordDataset(dataset_file).map(decode_func).batch(1).repeat(epochs)

def make_decoder(serialized_features):
    features_dict = {f: tf.io.VarLenFeature(dtype=tf.int64) for f in serialized_features}
    features_dict['label'] = tf.io.VarLenFeature(dtype=tf.int64)

    def decode(serialized_example):
        features = tf.io.parse_example(
            [serialized_example],
            features_dict
        )
        return features
    return decode

def select_features(features, hash_size, model_features):
    selected_features = []
    for feature_name in model_features:
        feature_tensor = features.get(feature_name[0])
        dense_feature_tensor = tf.transpose(tf.sparse.to_dense(tf.cast(feature_tensor, dtype=tf.int32)))
        dense_feature_tensor = tf.math.mod(dense_feature_tensor, hash_size)
        selected_features.append(dense_feature_tensor)
    return selected_features

def build_optimizer(learning_rate, optimizer):
    if optimizer == 'RMSProp':
        opt = tf.compat.v1.train.RMSPropOptimizer(learning_rate=learning_rate)
    elif optimizer == 'adagrad':
        opt = tf.compat.v1.train.AdagradOptimizer(learning_rate=learning_rate)
    elif optimizer == 'adam':
        opt = tf.compat.v1.train.AdamOptimizer(learning_rate=learning_rate)
    else:
        opt = tf.compat.v1.train.GradientDescentOptimizer(learning_rate=learning_rate)
    return opt

def predictions_saver(prediction_path, predictions):
    with open(prediction_path, "a") as output_file:
        output_file.write('\n'.join([str(x) for x in predictions]) + '\n')

def summarize_weights(session):
  if hasattr(session, 'raw_session'): session = session.raw_session()
  weights = session.run(tf.compat.v1.trainable_variables())
  summary = sum(map(lambda x: x.sum(), weights))
  print("Summary of weights: %.20f" % summary)

if __name__ == '__main__':
    dataset_file = sys.argv[1]
    prediction_path = sys.argv[2]
    header = list(string.ascii_uppercase) + list(string.ascii_lowercase) + [str(i) for i in range (10)]
    epochs = 1
    hash_size = 2 ** 25
    model_features = "A-B-C-D".split('-')
    init_std = 0.01
    embedding_size = 4
    l2_reg_lr, l2_reg_nn, l2_reg_emb = 0, 0, 0
    hidden_units_num = 200
    hidden_layers_num = 4
    dnn_activation_function = "relu"
    dnn_dropout = 0
    learning_rate = 0.0004
    optimizer = "adam"

    graph = tf.Graph()
    with graph.as_default():
        # Input pipeline
        iterator, training_init_op = input_pipeline(dataset_file, header, epochs)
        features = iterator.get_next()
        selected_features = select_features(features, hash_size, model_features)
        stacked_features = tf.stack(selected_features, axis=1)
        inputs = tf.reshape(stacked_features, [-1, len(model_features)])
        label = tf.reshape(tf.sparse.to_dense(features['label']), [-1])

        # Linear part
        emb_linear = tf.compat.v1.keras.layers.Embedding(
            hash_size,
            1,
            embeddings_initializer=RandomNormal(mean=0.0, stddev=init_std, seed=SEED),
            embeddings_regularizer=l2(l2_reg_emb),
            embeddings_constraint=None,
            mask_zero=False,
            input_length=len(model_features)
        )(inputs)

        linear_logit = Linear(l2_reg_lr)(emb_linear)

        # Embeddings for FM / DNN
        emb_fm = tf.compat.v1.keras.layers.Embedding(
            hash_size,
            embedding_size,
            embeddings_initializer=RandomNormal(mean=0.0, stddev=init_std, seed=SEED),
            embeddings_regularizer=l2(l2_reg_emb),
            embeddings_constraint=None,
            mask_zero=False,
            input_length=len(model_features)
        )(inputs)

        # FM part
        fm_logit = FM()(emb_fm)

        # DNN part
        dnn_input = Flatten()(emb_fm)
        dnn_output = DNN([hidden_units_num] * hidden_layers_num, activation=dnn_activation_function, l2_reg=l2_reg_nn,
                    dropout_rate=dnn_dropout, seed=SEED)(dnn_input)
        # dnn_logit = Dense(1, use_bias=False, activation=None, kernel_initializer="ones")(dnn_output)
        dnn_logit = Dense(1, use_bias=False, activation=None, kernel_initializer=GlorotUniform(seed=SEED))(dnn_output)

        # Add together
        final_logit = Add()([linear_logit, fm_logit, dnn_logit])

        # Prediction layer
        pred = tf.reshape(PredictionLayer('binary')(final_logit), [-1], name=prediction_output_name)
        loss = tf.identity(tf.compat.v1.losses.log_loss(label, pred), name='LOSS')
        opt = build_optimizer(learning_rate, optimizer)
        optimizer = opt.minimize(loss)

        init_all_vars = tf.compat.v1.global_variables_initializer()
        saver = tf.compat.v1.train.Saver()

        config = tf.compat.v1.ConfigProto()
        config.gpu_options.allow_growth = True
        session = tf.compat.v1.Session(graph=graph, config=config)

        session.run(init_all_vars)
        session.run(training_init_op)
        summarize_weights(session)

        while True:
            try:
                inp, predictions, _ = session.run([inputs, pred, optimizer])
                summarize_weights(session)

                with open(prediction_path, "a") as output_file:
                    output_file.write('\n'.join([str(x) for x in predictions]) + '\n')
            except tf.errors.OutOfRangeError:
                break

        session.close()

Could you help me to understand why I don't get the exact same predictions after consecutive runs ?

That's a very thorough report, @ornellamarciano; thank you. Having scanned your code, what stands out to me is your use of tf.compat.v1.keras.layers.Embedding. We have isolated a source of nondeterminism to the backprop into embedding matrices (in TensorFlow) and we're actively working on a patch to address this. We plan to release soon, at which point I'll ask you to test again with the patch.
Deeper information: the forward path of selecting word embeddings for a given batch is usually implemented with tf.gather (and I've confirmed that it is true for tf.compat.v1.keras.layers.Embedding). The backprop for tf.gather produces a tf.IndexedSlices gradient tensor which is reduced to a dense tensor, in order to update the embedding matrix, using tf.convert_to_tensor. tf.convert_to_tensor uses one of the segment sum ops, which we has been confirmed to operate nondeterministically. The upcoming patch, which we will be in version 0.4.0 of the framework-determinism package, and applied using fwd9m.tensorflow.enable_determinism(), will dynamically patch the segment sum ops to make them operate deterministically for 16-bit floating-point and 32-bit floating-point operands. The patch will not address nondeterminism in 64-bit floating-point operands; that will be addressed by a later CUDA-level fix (I don't know when we'll get to that).
Thanks to my colleague @wenscarl for helping to isolate the source and to develop the patch.

@duncanriach  @wenscarl Thanks a lot for your quick and detailed answer !
Regarding float64, I tried it to escape non-determinism due to a potential issue in floating point. But if with the new patch, I manage to get deterministic results, I will totally work with float32 (faster and less memory consumption).
Thanks again.

Right, I would expect you to see a much smaller amount of noise accumulating when using float64, which seems to be what you witnessed.

Hi @duncanriach do you know when the patch solving the non determinism of embedding layers will be released ? Thanks for your help!

I don't have an ETA for you, but I can tell you that we're actively working on it and that's it's top priority. I'm hoping for a release in the next few weeks.

Hi @duncanriach do you have any update ? Thanks a lot!

Hi @ornellamarciano,
Thanks for checking-in.
From stock TensorFlow version 2.5 onwards, the use of TensorFlow's segment reduction ops running on a GPU, with the expectation of reproducibility (when TF_DETERMINISTIC_OPS is set to '1' or 'true'), will cause tf.errors.UnimplementedError to be thrown, this includes back-propagating into a dense embedding matrix via the backwards path through tf.gather (which uses tf.math.unsorted_segment_sum). See stock TensorFlow pull request 47772 for more information.
@benbarsdell has developed deterministic GPU implementations of all the TensorFlow segment reduction ops, including tf.math.unsorted_segment_sum. A pull request to add this to stock TensorFlow will be created soon (there are some dependencies that it's waiting for). I expect these changes to be released in stock TensorFlow version 2.7.

You might also want to try applying the unreleased patch from the cloned repo. See the discussion in issue 19.


add a disaggregator to identify political-bias

@NimaBoscarino and I have been discussing potentially identifying political bias.
intuitively, I would say we can start with liberal vs. conservative.

After some internal discussions at Hugging Face, we figured that a safer option to start with would be "professions", so I've opened up #28 for that. It doesn't change anything API-wise though!


Tests should not depend on the status of the API

Tests are currently connecting to the actual Riot API in order to retrieve data. This dependency should be broken and the API should be mocked so that only this project's code is being tested.
Consequently, the API Key and Tournament API Key can then be removed from the RiotApiTest class.

Agreed. For unit tests this should be mocked.
Thanks for pointing this out.

I'm currently toying with this with this for self development, but not enough to commit to a PR. If I get there, I'll submit.

Sounds good. Feedback and PRs are always very much appreciated.

Another problem that came up while building with tests on is, that Tryndamere changed his summoner name.

K, I'll update it to use the summoner name "LoveMeSomeRegi".
Thanks for the heads up.

https://github.com/taycaldwell/riot-api-java/issues/83 has been updated to include the remaining work needed for this task.


Enable OGS Map Services Backend

(from GPV user group 2018)

I give this one a thumbs up! This would make it possible for GPV to be a an opensource system that can run completely on free opensource technology.


Iterating over sub-object

I'm sorry but I'm completely lost
I just want to iterate over a value:
    auto tiles = j.find("tiles");
    if (tiles != j.end())
    {
      for (auto tile : *tiles)
      {
        std::cout << tile.key() << std::endl;
        //printf("Name: %s\n", tile.key().c_str());
      }
    }

I even used std::cout just like the examples.
{
  "tiles":
  {
    "a": [2, 5],
    "b": [3, 0],

What am I doing wrong? It says error: no member named 'key' in tile but that makes no sense because im *iterator so I should already be ranged-for-ing over each tile.
The examples also don't really provide any insight into iterating over things like these.
Iterating over the sub-object and then std::cout << tile << std::endl; works like expected, however that is just magic. I want to print the key and the value separately, one as a string and the other as an array using indexing operator.

You may want to look at iteration wrappers - here is an example how to use them https://github.com/nlohmann/json/blob/develop/doc/examples/iterator_wrapper.cpp.

Thanks, worked :)


Rejigger How-To information

[x] Remove optional HTML5 elements (header, footer, main) from HTML boilerplate to trim it down.
[x] Remove incorrect blog.css link from top of index.html.
[ ] Do we actually need all three levels of div tag? Seems a bit much.

the three levels of div are as Lou defined "predictable spacing and layout". We would prefer people follow that and add their stuff inside those divs. I'll add a page structure drawing under the snippet to explain that better. :)

Heh, okay.

https://github.com/mapzen/styleguide/pull/168 @louh all yours!


Moved attestation factory back to static method on attestation class …

…and return a concrete type not a pointer
Pull Request Checklist
Please leverage this checklist as a reminder to address commonly occurring feedback when submitting a pull request to make sure your PR can be reviewed quickly:
See the detailed list in the contributing guide.

[X] C++ Guidelines
[X] Doxygen docs
[X] Unit tests
[X] No unwanted commits/changes
[X] Descriptive title/description

[X] PR is single purpose
[] Related issue listed

[X] Comments in source
[X] No typos
[X] Update changelog
[X] Not work-in-progress
[X] External references or docs updated
[X] Self review of PR done
[X] Any breaking changes?

/azp run prepare-pipelines

Tagging @JeffreyRichter and @antkmsft since we were talking about this PR yesterday.

/azp run cpp - attestation

/check-enforcer evaluate


Remove osm-arc validations

This checklist is used to make sure that common guidelines for a pull request are followed.
Related command
az k8s-extension create --cluster-name  --resource-group  --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster -release-train stable --name osm --release-namespace arc-osm-system --version 1.2.9 --auto-upgrade-minor-version false
General Guidelines

[ ] Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
[ ] Have you run python scripts/ci/test_index.py -q locally? (pip install wheel==0.30.0 required)
[X] My extension version conforms to the Extension version schema

For new extensions:

[X] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

what is the reason for removing this validation?  Is it not applicable anymore?  Can you pl. update the PR description with details?

@nshankar13,  what is the reason for removing this validation?  Is it not applicable anymore?  Can you pl. update the PR description with details?


Welcome Screen

Done

Edited the App.tsx to add the homestack, test screen (keeping a test screen to test out new components)
Added leftBackArrow icon pick and the welcome screen pic to assets
Welcome Screen complete
Buttons navigates to Login Screen and SignUp Screen
Added test files for each new file.

To Do

Write functional test files instead of snap shots for each test files

Made changes to remove the test screen and route. Merging and deleting branch.


Add generic workflow for un-triggered actions

Context:
PR #166 refactored CI checks leading to some stale check that prevent some PRs from merging.
Description of the Change:
Due to path filtering, a pull request that points to the develop branch of the repo will not trigger the build workflow on builds.yml and then workflows will stay stale and prevent merging. This PR adds a generic workflow, with the same name, will return true in any case the build workflow in builds.yml` is not triggered.
See github actions docs here.
Benefits:
Avoid blocking merges for no reason.
Possible Drawbacks:
None

Closing this as I found a cleaner way to do it via github config


docs: Unified envoy gateway installation documentation

What type of PR is this?
Fixes #2616

@shawnh2 pls refer to https://github.com/envoyproxy/gateway/pull/2621. 🤔


matrix assignment with inconsistent size

Summary:
I just realized I can do assignment to a matrix of different size in transformed param block. I'm actually not sure if this is by design.
Description:
See below.
Reproducible Steps:
Using 2.28.1, I can do this
transformed parameters {
  matrix[2,2] x = [[-4.6, 0.0], [0.0,-2.8]];
  matrix[1,1] y = x;
  print(y[1,1], " ", y[2,2]);
}

The output is
//....
Model contains no parameters, running fixed_param sampler, no updates to Markov chain
-4.6 -2.8

-4.6 -2.8
//...

Doing the same in transformed data block causes runtime error, as I expect
transformed data {
  matrix[2,2] x = [[-4.6, 0.0], [0.0,-2.8]];
  matrix[1,1] y = x;
  print(y[1,1], " ", y[2,2]);
}

The output:
Assertion failed: (rows == this->rows() && cols == this->cols() && "DenseBase::resize() does not actually allow to resize."), function resize, file stan/lib/stan_math/lib/eigen_3.3.9/Eigen/src/Core/DenseBase.h, line 257.

Expected Output:
Assertion failure error.
Current Version:
v2.28.1

This has been a long standing issue. There is an issue open in stanc3 for it: https://github.com/stan-dev/stanc3/issues/523 but we have since established its not the fault of stanc3. I believe there is only a check missing in https://github.com/stan-dev/stan/blob/develop/src/stan/model/indexing/assign.hpp
I think no one wants to touch this one because there is a never confirmed (I think it was even denied on multiple occasions) that this bug is abused somewhere in rstanarm.

@bgoodri:  can you confirm exactly which piece of the error checking for sizes you need us not to fix (until we get ragged arrays)?

Ben seems to think it's not him using it: https://github.com/stan-dev/stanc3/issues/523#issuecomment-617462065
I think it wouldn't be too unreasonable to patch it in a branch and then test RStan, brms, and rstanarm against it now that we have the ability to use RStan on newer versions?

I doubt anyone is using that on purpose.
On Fri, Nov 19, 2021 at 12:25 PM Brian Ward @.***>
wrote:

Ben seems to think it's not him using it: stan-dev/stanc3#523 (comment)
https://github.com/stan-dev/stanc3/issues/523#issuecomment-617462065
I think it wouldn't be too unreasonable to patch it in a branch and then
test RStan, brms, and rstanarm against it now that we have the ability to
use RStan on newer versions?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/3084#issuecomment-974263100, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZ2XKRB452PXLT3SKEAYGDUM2CADANCNFSM5IIADU4A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


NoMethodError:   undefined method `orm=' for DatabaseCleaner:Module

To be right on point, here's my spec_helper.rb:
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../config/environment', __dir__)
require 'mongoid'
require 'rspec/rails'
require 'factory_bot_rails'
require 'rails/mongoid'
require 'database_cleaner'
require 'mongoid-rspec'

Mongoid.load!(Rails.root.join('config', 'mongoid.yml'))

Rails.backtrace_cleaner.remove_silencers!

# Load support files
Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].sort.each { |f| require f }
RSpec.configure do |config|
  config.mock_with :rspec
  config.infer_base_class_for_anonymous_controllers = false
  config.order = 'random'

  config.before(:suite) do
    DatabaseCleaner.strategy = :truncation
    DatabaseCleaner.orm = 'mongoid'
  end

  config.before(:each) do
    DatabaseCleaner.clean
  end
end

When I run bundle exec rspec, it throws:
An error occurred in a `before(:suite)` hook.
Failure/Error: DatabaseCleaner.orm = 'mongoid'

NoMethodError:
  undefined method `orm=' for DatabaseCleaner:Module
# ./spec/spec_helper.rb:23:in `block (2 levels) in <top (required)>'

Hi! Please downgrade to database_cleaner v1.99 and resolve all the
deprecation warnings. Then, you can upgrade to v2.0.
On Wed, Feb 3, 2021 at 8:07 AM João Victor Ramires Guimarães Brito <
<EMAIL_ADDRESS>wrote:

To be right on point, here's my spec_helper.rb:
ENV['RAILS_ENV'] ||= 'test'require File.expand_path('../config/environment', dir)require 'mongoid'require 'rspec/rails'require 'factory_bot_rails'require 'rails/mongoid'require 'database_cleaner'require 'mongoid-rspec'
Mongoid.load!(Rails.root.join('config', 'mongoid.yml'))
Rails.backtrace_cleaner.remove_silencers!
Load support filesDir["#{File.dirname(FILE)}/support/**/*.rb"].sort.each { |f| require f }RSpec.configure do |config|
config.mock_with :rspec
config.infer_base_class_for_anonymous_controllers = false
config.order = 'random'
config.before(:suite) do
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.orm = 'mongoid'
end
config.before(:each) do
DatabaseCleaner.clean
endend
When I run bundle exec rspec, it throws:
An error occurred in a before(:suite) hook.
Failure/Error: DatabaseCleaner.orm = 'mongoid'
NoMethodError:
undefined method orm=' for DatabaseCleaner:Module# ./spec/spec_helper.rb:23:in block (2 levels) in <top (required)>'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/DatabaseCleaner/database_cleaner/issues/672, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAEH32MRWFFBRVCTVC27F3S5FYFHANCNFSM4XBDUB7A
.

@botandrose Thank you. It worked. Closing issue.


messages

closes #390
@h1alexbel take a look, please

PR-Codex overview
This PR focuses on improving the code readability and providing more descriptive error messages.
Detailed summary

Improved readability by adding line breaks and indentation.
Added descriptive error messages for better understanding of exceptions thrown.
Refactored code to use formatted strings for better readability.
Fixed imports and removed unused code.

The following files were skipped due to too many changes: src/test/java/io/github/eocqrs/kafka/consumer/KfConsumerTest.java, src/test/java/io/github/eocqrs/kafka/fake/FkConsumerTest.java

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@l3r8yJ thank you for your Pull Request. I'll assign someone to review it soon.
If this PR solves a todo from the code, please don't forget to remove it.

@h1alexbel please review this Pull Request. Deadline (when it should be merged or closed) is 2023-07-11T20:09:04.552438.
You should check if the requirements have been implemented (partially or in full), if there are unit tests covering the changes and if the CI build passes. Feel free to reject the PR or ask for changes if it's too big or not clear enough.
Estimation here is 30 minutes, that's how much you will be paid. You will be paid even if this PR gets rejected.

@h1alexbel reminder

@h1alexbel fixed, take a look, please
need to merge before #410

@h1alexbel Don't forget to close this ticket before the deadline (2023-07-11T20:09:05). You are past the first half of the allowed period.

@rultor merge

@rultor merge

@h1alexbel OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@h1alexbel Done! FYI, the full log is here (took me 2min)

@l3r8yJ thanks

@h1alexbel thank you for resolving this ticket. I've just added it to your active invoice. You can always check all your invoices and more on the Contributor Dashboard.


BUG: pip install SciPy fails under PyPy

Describe your issue.
I'm seeing this in SymPy CI in https://github.com/sympy/sympy/pull/23859. I don't immediately have a computer on which I can test this with PyPy locally.
Essentially pip install scipy has failed under PyPy with:
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/_tags.py", line 79, in __init__
          f'Unknown Python implementation: {self.implementation}. '
      NotImplementedError: Unknown Python implementation: pypy38. Please report this to https://github.com/FFY00/mesonpy/issues and include information about the Python distribution you are using.

You can see the CI setup for this up to the line that fails here:
https://github.com/sympy/sympy/blob/7754d037d97d56de9e1bcecb81ecadd4a2e683be/.github/workflows/runtests.yml#L170-L208
The logs for the failed SciPy build are here:
https://github.com/sympy/sympy/runs/7583332309?check_suite_focus=true
I have just confirmed that the same CI setup with scipy!=1.9.0 builds 1.8.1 just fine so it seems to be a problem with the 1.9.0 release.
Reproducing Code Example
# Run on Ubuntu 20.04 under PyPy 3.8.
# There might be more to it than this.
# See the CI config above.
pip install scipy

Error message
+ meson install --destdir /tmp/pip-install-pkqrdcy5/scipy_8a1b991ddd494020a114650533cb4ba6/.mesonpy-ve4kjypi/install
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/PyPy/3.8.13/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 156, in prepare_metadata_for_build_wheel
          hook = backend.prepare_metadata_for_build_wheel
      AttributeError: module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 714, in _calculate_file_abi_tag_heuristic_posix
          return mesonpy._tags.StableABITag(tag)
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/_tags.py", line 39, in __init__
          raise ValueError(f'Invalid PEP 3149 stable ABI tag, expecting pattern `{self._REGEX.pattern}`')
      ValueError: Invalid PEP 3149 stable ABI tag, expecting pattern `^abi(?P<abi_number>[0-9]+)$`

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/opt/hostedtoolcache/PyPy/3.8.13/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/opt/hostedtoolcache/PyPy/3.8.13/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/hostedtoolcache/PyPy/3.8.13/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 160, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 927, in build_wheel
          return project.wheel(out).name
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 875, in wheel
          wheel = _WheelBuilder(self).build(self._install_plan, self._copy_files, self._build_dir)
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 356, in build
          wheel_file = pathlib.Path(directory, f'{self.name}.whl')
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 165, in name
          python_tag=self._project.python_tag,
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 657, in python_tag
          selected_tag = self._select_abi_tag()
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 779, in _select_abi_tag
          tags = self._files_by_tag()
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 759, in _files_by_tag
          tag = self._calculate_file_abi_tag_heuristic(file)
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 723, in _calculate_file_abi_tag_heuristic
          return self._calculate_file_abi_tag_heuristic_posix(filename)
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 716, in _calculate_file_abi_tag_heuristic_posix
          return mesonpy._tags.LinuxInterpreterTag(tag)
        File "/tmp/pip-build-env-mupbdlja/overlay/lib/pypy3.8/site-packages/mesonpy/_tags.py", line 79, in __init__
          f'Unknown Python implementation: {self.implementation}. '
      NotImplementedError: Unknown Python implementation: pypy38. Please report this to https://github.com/FFY00/mesonpy/issues and include information about the Python distribution you are using.

SciPy/NumPy/Python version information
SciPy 1.9.0, PyPy 3.8

I think this bug should be filed against https://github.com/FFY00/mesonpy, as the message says.

Let's leave this open too though. I believe there are more issues with PyPy, and we don't have any CI for PyPy - but if pip install scipy on PyPy worked for you before, we should get things back to that same state for 1.9.x

I think this bug should be filed against https://github.com/FFY00/mesonpy, as the message says.

I opened an issue with mesonpy: https://github.com/FFY00/meson-python/issues/121
I think that "can't install scipy under pypy" is a SciPy issue in any case though even if the root cause lies elsewhere.

I believe there are more issues with PyPy, and we don't have any CI for PyPy

Is PyPy not considered to be "supported" by SciPy? The release notes say "For running on PyPy, PyPy3 6.0+ is required" which suggests that it is supported in which case perhaps some CI or pre-release testing would make sense.

PyPy is at the same status as other Python implementations (Graal, Pyston, etc.) and platforms (AIX, HPC platforms, the niche Debian architectures) for which we have no CI: we accept patches and do our best to be responsive to issues, but without CI that's all we can do. So "tier 3 support" I guess.
We had PyPy in CI for a little while (IIRC ~1.5 years back), but it was too unstable so we removed it again.

"tier 3 support" I guess.

Maybe there could be a very low level of platform CI that just tests if you can build, install, import a few things and call a couple of functions. They could just be used on the release branches rather than main so you can see around release time if any platforms are likely affected by the changes since the last release.
The change here seems to be that SciPy is not usable under PyPy at all now if I've understood this correctly. Maybe that's still a valid choice as part of the move to meson but it's one of those things that you'd want to know about and mention in the release notes etc.

We had PyPy in CI for a little while (IIRC ~1.5 years back), but it was too unstable so we removed it again.

Maybe it's different at the C level. For SymPy which is pure Python PyPy used to show instability and bugs but that has stabilised and it's unusual now to see any PyPy specific bugs or even differences (the only example I can think of right now is https://github.com/sympy/sympy/pull/23321#issuecomment-1166782369).
Recently most instability in SymPy's CI seems to be SciPy-related :)

BTW the bug in question seems to be in the PEP 517 build backend glue, not Meson itself. There's some testing at https://github.com/mattip/scipy/pull/1 but that uses Meson directly and/or dev.py
@mattip has contributed a couple pypy fixes for Meson and helped us set up our CI so that we are testing our "basic python project" CI on pypy as well as cpython. :)

Maybe it's different at the C level.

Yes, it was mostly related to the cpyext layer IIRC.

The change here seems to be that SciPy is not usable under PyPy at all now if I've understood this correctly. Maybe that's still a valid choice

It's simply a bug, not a deliberate choice.

Maybe there could be a very low level of platform CI that just tests if you can build, install, import a few things and call a couple of functions. They could just be used on the release branches

Yes, I think we can add such a job. It should run on main too (perhaps not PRs, but at least on merges to main so that we catch regressions in time).

There's now also PyPy CI in meson-python, including a "scipy-like" package for which wheels are built and checked. The problems here are fixed, so I'll close the issue..

Good work all!

How do I get the fix?
I am trying to install scipy and I am currently getting this error:
AttributeError: module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'
This appears to be the same as what was originally reported. I'm using PyPy version 7.3.9 which is the most recent version on https://www.pypy.org/download.html and I just upgraded pip using pypy3.exe -m pip install --upgrade pip.
Is there anything else I need to update to install scipy under pypy? It installed just fine under my normal python3 installation.

@snydej can you please open a new issue with the full traceback you are seeing, and Cc me on the issue?

AttributeError: module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'

This is a generic message that will be visible for all build failures, the actual problem will be higher up.
The problem reported in this bug reported (NotImplementedError: Unknown Python implementation: pypy38) was fixed. Maybe there's another issue on your system, or with PyPy in general.

@snydej: perhaps try the new 7.3.10 release candidates. I think you will have a smoother user experience using conda with PyPy, since you will get pre-compiled binary packages.

@snydej: perhaps try the new 7.3.10 release candidates. I think you will have a smoother user experience using conda with PyPy, since you will get pre-compiled binary packages.

Hi mattip
I have tried to install scipy 1.9.3 via pip with pypy 7.3.10 without success. I still saw "module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'"

scipy 1.8.1 is fine.

Is there a known workaround for this? Setting scipy==1.8.1 didn't help me. I'm trying to install librosa which depends on scipy.

Maybe this should be reopened.
After opening this issue I pinned scipy<1.9.0 in the relevant part of SymPy's CI. I just tried removing that pin but found that SciPy 1.9.3 fails to build under PyPy: https://github.com/sympy/sympy/pull/24393
The build failure log ends with a segfault in boost.npz:
[237/1576] Generating scipy/special/_data_boost with a custom command
      FAILED: scipy/special/boost.npz
      /opt/hostedtoolcache/PyPy/3.8.15/x64/bin/python ../../scipy/special/utils/makenpz.py --use-timestamp boost -o scipy/special
      Segmentation fault (core dumped)
      [238/1576] Generating 'scipy/special/_ufuncs.pypy38-pp73-x86_64-linux-gnu.so.p/_ufuncs.c'.
      [239/1576] Generating 'scipy/special/cython_special.pypy38-pp73-x86_64-linux-gnu.so.p/cython_special.c'.
      ninja: build stopped: subcommand failed.
      INFO: autodetecting backend as ninja
      INFO: calculating backend command to run: /tmp/pip-build-env-dtk8i39b/normal/bin/ninja
      + meson setup --prefix=/opt/hostedtoolcache/PyPy/3.8.15/x64 /tmp/pip-install-pzj_psgt/scipy_7357d47d5[480](https://github.com/sympy/sympy/actions/runs/3707780050/jobs/6284631258#step:8:481)420c8ee98ac8970ba85b /tmp/pip-install-pzj_psgt/scipy_7357d47d5480420c8ee98ac8970ba85b/.mesonpy-5x6cfkno/build --native-file=/tmp/pip-install-pzj_psgt/scipy_7357d47d5480420c8ee98ac8970ba85b/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
      + meson compile
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/PyPy/3.8.15/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 144, in prepare_metadata_for_build_wheel
          hook = backend.prepare_metadata_for_build_wheel
      AttributeError: module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/opt/hostedtoolcache/PyPy/3.8.15/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
          main()
        File "/opt/hostedtoolcache/PyPy/3.8.15/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/hostedtoolcache/PyPy/3.8.15/x64/lib/pypy3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 148, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
        File "/tmp/pip-build-env-dtk8i39b/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 1061, in build_wheel
          return project.wheel(out).name
        File "/tmp/pip-build-env-dtk8i39b/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 920, in wheel
          wheel = self._wheel_builder.build(self._build_dir)
        File "/tmp/pip-build-env-dtk8i39b/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line [502](https://github.com/sympy/sympy/actions/runs/3707780050/jobs/6284631258#step:8:503), in build
          self._project.build()  # ensure project is built
        File "/opt/hostedtoolcache/PyPy/3.8.15/x64/lib/pypy3.8/functools.py", line 566, in wrapper
          result = user_function(*args, **kwds)
        File "/tmp/pip-build-env-dtk8i39b/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 737, in build
          self._meson('compile', *self._meson_args['compile'],)
        File "/tmp/pip-build-env-dtk8i39b/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 657, in _meson
          return self._proc('meson', *args)
        File "/tmp/pip-build-env-dtk8i39b/overlay/lib/pypy3.8/site-packages/mesonpy/__init__.py", line 652, in _proc
          subprocess.check_call(list(args), env=self._env)
        File "/opt/hostedtoolcache/PyPy/3.8.15/x64/lib/pypy3.8/subprocess.py", line 364, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['meson', 'compile']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

I don't know what causes it but something should really be done about the error message:
     AttributeError: module 'mesonpy' has no attribute 'prepare_metadata_for_build_wheel'

Failure should be reported in some other way than AttributeError.

I don't know what causes it but something should really be done about the error message:

https://github.com/mesonbuild/meson-python/pull/231

Since the underlying error is:
/opt/hostedtoolcache/PyPy/3.8.15/x64/bin/python \
    ../../scipy/special/utils/makenpz.py \
    --use-timestamp \
    boost \
    -o scipy/special

And this same program is run, more or less, in setup.py, it seems something rather fishy is going on. But since that script does pretty much nothing other than numpy.loadtxt() and numpy.savez_compressed() it sounds like whatever it is, is probably a numpy bug, not a SciPy one.

The build failure log ends with a segfault in boost.npz:

That's a segfault of PyPy it looks like.

Maybe this should be reopened.

We can, but I'm not sure there is much point in doing that (EDIT: I'll reopen, but see below). PyPy isn't very stable, not tested in SciPy CI, and building SciPy from source with PyPy is even worse. As @mattip said above (https://github.com/scipy/scipy/issues/16737#issuecomment-1329338815), the experience with conda-forge may be better - and at least maintainable. Help may be needed there too, but it's tractable.  See https://www.pypy.org/posts/2022/11/pypy-and-conda-forge.html.
Building from source with such a complex and untested stack is just not a great idea. We're happy to take patches, whether this issue is open or not. But just reopening it isn't going to fix anything I'm afraid.
Recommendations for anyone using PyPy:

Use conda-forge to install PyPy and the dependencies you need
If there's no PyPy package for your platform or it's outdated, go help out to build it at https://github.com/conda-forge/scipy-feedstock
If it turns out it needs a patch in either PyPy or SciPy, add it there and then upstream it (no need to wait for a new SciPy or PyPy release!)

PyPy isn't very stable, not tested in SciPy CI

We discussed above adding something to SciPy CI.
This issue was closed but did any version of SciPy correctly install? I previously confirmed that 1.9.0 and 1.9.1 did not install and I now confirm that 1.9.3 does not. SciPy 1.8.1 (pre meson) installed fine.

Again, it seems like Meson is a red herring here. The same script is run with setup.py or meson, so it should segfault the same way -- inside numpy.
Does SciPy 1.9.3 build with setup.py when run manually, without pip, but otherwise in the same environment? Alternatively, patch pyproject.toml so pip uses the setup.py backend?

Just that it builds, yes (on Linux, easiest case )

That is the part that this issue concerns and also the part that is still not currently working.

it seems like Meson is a red herring here.

Possibly. We are talking about build failures dating back to a release that involves a major change of build system though. It's possible that a change in numpy or something else is related to this but I think it would be quite naive to rule out anything related to the change of build system before the cause is understood. As I understood it there was an a priori expectation that some problems of this kind would emerge after making the change so it shouldn't be a surprise that when such problems arise the change of build system is considered as a potential cause. Of course that doesn't mean that the bug is in Meson itself but the general change in SciPy's build system should not be overlooked.
A final point: SymPy has no particular interest in this particular PyPy+SciPy combination but it just happens to be tested in CI. On the assumption that SciPy folks do want SciPy to be installable under PyPy I am taking the time to feed these issues back. If this particular setup is considered to be unsupported by SciPy then I'll make sure that SymPy does not test it any more and I won't open issues along the lines of this one.

Sure, but that's why my own suggestion (at least as a drive-by contributor) was just "to try to factor out the build system as a potential cause, it would be good to check if the issue reproduces when using setup.py".
I'm not suggesting to ignore the issue, merely to run more tests.

I'm not suggesting to ignore the issue, merely to run more tests.

I understand but it's not hard for anyone else to reproduce this setup and run whatever tests. Just remove the pin here (probably a much simpler config will reproduce this):
https://github.com/sympy/sympy/blob/4801871259a1a8db27bae99f5ea4db109bb1320a/.github/workflows/runtests.yml#L197-L201
Personally I have other things I need to spend time on. As I said above this setup is not needed by SymPy: I am just reporting what I observe in case it is of interest.

That said, I would not recommend any project to build SciPy from source on every run in their regular CI, not for any platform or interpreter.

Okay, well I'll keep it there for now until we can see that this is fixed but then I'll remove it.

Maybe at some point SciPy will release wheels for PyPy which would make testing downstream projects like sympy simpler.

In SymPy's case in particular I doubt that testing this setup does much besides detecting upstream bugs. You can question the value of that but apparently it picks up things that are not tested in upstream CI e.g. because SciPy CI doesn't have anything checking the PyPy build itself.

I can confirm that with the latest release of PyPy (7.3.11) this is now fixed as tested in this PR: https://github.com/sympy/sympy/pull/24393.

Today I installed scipy & scikit-learn successfully with pypy 7.3.11
Thanks.

getting this error when running in ubuntu-latest in github action
× Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [67 lines of output]
      The Meson build system
      Version: 0.62.2
      Source dir: /tmp/pip-install-ll2hfubu/scipy_2c9b53b7e8054244987219dc55724edb
      Build dir: /tmp/pip-install-ll2hfubu/scipy_2c9b53b7e8054244987219dc55724edb/.mesonpy-n6mu_qg8/build
      Build type: native build
      Project name: SciPy
      Project version: 1.9.0
      C compiler for the host machine: cc (gcc 10.2.1 "cc (Debian 10.2.1-6) 10.2.1 20210110")
      C linker for the host machine: cc ld.bfd 2.35.2
      C++ compiler for the host machine: c++ (gcc 10.2.1 "c++ (Debian 10.2.1-6) 10.2.1 20210110")
      C++ linker for the host machine: c++ ld.bfd 2.35.2
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Compiler for C supports arguments -Wno-unused-but-set-variable: YES
      Library m found: YES

This doesn't tell us much -- it seems like the second half of the build logs was accidentally omitted from your report, including the half with the error in it.

So I got this error message from running in github actions using scipy == 1.7.3 in requirements.txt @eli-schwartz

The github action environment was ubuntu-latest and it is an x86 based architecture

If it was a github actions run, can you give a link to the log?

https://github.com/DSGT-DLP/Deep-Learning-Playground/actions/runs/4299741691/jobs/7495260827

@andyfaff

@karkir0003 you missed the important bits of the build log out. You need to have a fortran compiler installed to build scipy.
× Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [67 lines of output]
      The Meson build system
      Version: 0.62.2
      Source dir: /tmp/pip-install-ll2hfubu/scipy_2c9b53b7e8054244987219dc55724edb
      Build dir: /tmp/pip-install-ll2hfubu/scipy_2c9b53b7e8054244987219dc55724edb/.mesonpy-n6mu_qg8/build
      Build type: native build
      Project name: SciPy
      Project version: 1.9.0
      C compiler for the host machine: cc (gcc 10.2.1 "cc (Debian 10.2.1-6) 10.2.1 20210110")
      C linker for the host machine: cc ld.bfd 2.35.2
      C++ compiler for the host machine: c++ (gcc 10.2.1 "c++ (Debian 10.2.1-6) 10.2.1 20210110")
      C++ linker for the host machine: c++ ld.bfd 2.35.2
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Compiler for C supports arguments -Wno-unused-but-set-variable: YES
      Library m found: YES

      ../../meson.build:41:0: ERROR: Unknown compiler(s): [['gfortran'], ['flang'], ['nvfortran'], ['pgfortran'], ['ifort'], ['g95']]
      The following exception(s) were encountered:
      Running "gfortran --version" gave "[Errno 2] No such file or directory: 'gfortran'"
      Running "gfortran -V" gave "[Errno 2] No such file or directory: 'gfortran'"
      Running "flang --version" gave "[Errno 2] No such file or directory: 'flang'"
      Running "flang -V" gave "[Errno 2] No such file or directory: 'flang'"
      Running "nvfortran --version" gave "[Errno 2] No such file or directory: 'nvfortran'"
      Running "nvfortran -V" gave "[Errno 2] No such file or directory: 'nvfortran'"
      Running "pgfortran --version" gave "[Errno 2] No such file or directory: 'pgfortran'"
      Running "pgfortran -V" gave "[Errno 2] No such file or directory: 'pgfortran'"
      Running "ifort --version" gave "[Errno 2] No such file or directory: 'ifort'"
      Running "ifort -V" gave "[Errno 2] No such file or directory: 'ifort'"
      Running "g95 --version" gave "[Errno 2] No such file or directory: 'g95'"
      Running "g95 -V" gave "[Errno 2] No such file or directory: 'g95'"

so is there a command I can run to install fortran compiler from within a docker image? @andyfaff

Apt install gfortran?

ok. Giving that a shot. For reference, I was using this base docker image: FROM nikolaik/python-nodejs:latest

Compiling scipy for CPython on Ubuntu is quite straightforward. However, PyPy is not currently actively tested/built/used by many of the scipy team, so it's not clear how you'll get on.

Fortunately, @karkir0003 appears to be using CPython, unlike the topic of this bug report.

@eli-schwartz Got this issue resolved. I changed the base docker image to a different version and it worked. I think that the base docker image used in the github action was 3.11, but scipy 1.7.3 didn't seem to be compatible with those settings, so I adjusted accordingly

@karkir0003 I'm general it's a bad idea to piggyback on another issue because it muddies the original conversation, and it confuses those trying to help. Advice on building would be better asked via the slack channel or the scipy-dev mailing list.

As far as I can tell this issue is fixed so I'm going to close this. Please let me know if I've overlooked something.

I noted above that it was fixed: https://github.com/scipy/scipy/issues/16737#issuecomment-1369250812

Also thanks to all from SciPy and PyPy for fixing!


Add CacheInvalidateCallback

We use redis client caching with BCAST so we have to implement custom mechanism.
I also added p.background() because otherwise it didn't get called.

Hi @altanozlu, thank your for your PR. The PR looks good to me. I just feel CacheInvalidateCallback is a little bit too long. Maybe OnInvalidations is enough?
Another thing is that I don’t understand how this callback mechanism related to BCAST client-side caching. Would you mind elaborate how do you use BCAST currently and how will you use this callback with it?

I use BCAST to listen for prefixes, but I don't use get directly on current connection, another service uses get, set, del. I use custom caching mechanism and I need to know if the key is invalidated or not.

Hi @altanozlu, that is interesting. So you just need those notifications without caching values in memory of client side? Redis has another feature called keyspace notification. Maybe it also suite for your need.

Hi @rueian, thanks for letting me know about keyspace notifications i'll check it. Actually we cache value but not the value in redis. It's like i call grpc.helloworld() and server uses 3 different redis get, the grpc client gets key information from headers and says this response mapped to this keys. when any of these 3 keys invalidate we invalidate the response.


Makes the global remap data structure ref-counted

This solves a number of issues, such as:
 1. Remap plugins won't have to refcount itself
 2. We can, I think, make remap plugins reloadable

The code didn't turn out quite as I wanted; I had initially
intended to store the leased table in the t_state (::State)
object, but circular references here made that difficult
without major refactoring. Instead, I hoisted it up to the
HttpSM, and added a UrlRewrite* argument to a couple of
functions that needs it in addition to the t_state.

@SolidWallOfCode What say ye? Should we do this? Once this is landed, I'd also like to remove the ref-counting that's done in various plugins.


Issue with SSL support in the official docker image

I have an issue with enabling of SSL support on postgres docker image. The host is Window 10 machine. The following configuration is used in docker-compose.yml.
version: '3.5'
services:
  postgresserver:
    image: postgres:14.5
    container_name: postgresserver
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: my_password
      PGPORT: 5432
      command: -c ssl=on -c ssl_cert_file=/var/lib/postgresql/server.crt -c ssl_key_file=/var/lib/postgresql/server.key -c ssl_ca_file=/var/lib/postgresql/CA.pem -c clientcert=verify-ca
    volumes:
      - "./certs/myCA.pem:/var/lib/postgresql/CA.pem"
      - "./certs/postgresserver.internal.crt:/var/lib/postgresql/server.crt"
      - "./certs/postgresserver.internal.key:/var/lib/postgresql/server.key"
    networks:
      default:
        aliases:
          - postgresserver.internal
  openssl:
    image: shamelesscookie/openssl:1.1.1
    container_name: openssl
    stdin_open: true
    tty: true

networks:
  default:
    name: dummy network
    driver: bridge
    ipam:
      config:
        - subnet: <IP_ADDRESS>/16

The files server.crt, server.key contain the server certificate and the private key signed by my own CA athorities whose certificate is in CA.pem
I have tried to connect using the pre-installed psql-client from Windows PowerShell on host as follows:
& 'C:\Program Files\PostgreSQL\14\bin\psql.exe' "sslmode=require host=localhost port=5432 dbname=test"

This call have produced the following output:
psql: error: connection to server at "localhost" (::1), port 5432 failed: server does not support SSL, but SSL was required

The call without "sslmode=require" switch works like a charm.
I have also tried to use openssl from openssl container as follows:
openssl s_client -starttls postgres -connect postgresserver:5432

This call has produced the following output:
CONNECTED(00000003)
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 1 bytes and written 8 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

Your command: line is indented two spaces too far. I also ran into [42] FATAL:  unrecognized configuration parameter "clientcert" so I removed that, and I think the server.pem isn't necessary. Also ensure you have proper permissions on the files: server.crt is chmod 777. server.key is chown 999 and chmod 600
$ cat docker-compose.yml
version: '3.5'
services:
  postgresserver:
    image: postgres:14.5
    container_name: postgresserver
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: my_password
      PGPORT: 5432
    command: -c ssl=on -c ssl_cert_file=/var/lib/postgresql/server.crt -c ssl_key_file=/var/lib/postgresql/server.key
    volumes:
      - ./server.crt:/var/lib/postgresql/server.crt
      - ./server.key:/var/lib/postgresql/server.key

$ ls -ln
total 12
-rw-r--r-- 1 1000 1000  444 Aug 22 16:57 docker-compose.yml
-rwxrwxrwx 1    0 1000 1939 Aug 22 16:01 server.crt
-rw------- 1<PHONE_NUMBER> Aug 22 16:00 server.key

$ docker-compose up -d
Creating network "postgres_default" with the default driver
Creating postgresserver ... done

$ docker-compose logs | tail -n 10
postgresserver    | server stopped
postgresserver    |
postgresserver    | PostgreSQL init process complete; ready for start up.
postgresserver    |
postgresserver    | 2022-08-22 16:59:21.528 UTC [1] LOG:  starting PostgreSQL 14.5 (Debian 14.5-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
postgresserver    | 2022-08-22 16:59:21.529 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
postgresserver    | 2022-08-22 16:59:21.529 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgresserver    | 2022-08-22 16:59:21.531 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresserver    | 2022-08-22 16:59:21.535 UTC [59] LOG:  database system was shut down at 2022-08-22 16:59:21 UTC
postgresserver    | 2022-08-22 16:59:21.541 UTC [1] LOG:  database system is ready to accept connections

$ docker run -it --network=host --rm postgres bash
root@demo:/# psql "sslmode=require host=localhost user=postgres password=my_password"
psql (14.5 (Debian 14.5-1.pgdg110+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

postgres=#

Closing assuming this is resolved


Package no longer works, consider deprecation.

Hello! Thank you for including Gleam in Chocolatey!
Unfortunately this package is now very out of date and no longer works after a fresh install. Because of this we in the Gleam community are getting almost daily complaints from people thinking that the language doesn't work due to this broken package, which is both causing support work and giving people a very poor first impression of the project.
The person who was previously maintaining this package is no longer able to do so as he no longer has a Windows computer.
Would we be able to deprecate or otherwise remove this package, or at least the already released versions? This would make things easier for us in the Gleam team.
Thanks,
Louis

Sorry gang, I've only just realised who is maintaining this now. I thought it was the Chocolatey people


[Feature]: Allow PackageSourceMapper to report sources that are unused add configure the minimal set

NuGet Product(s) Involved
Other/NA
The Elevator Pitch
I used the PackageSourceMapper tool on NuGet/Insights. Unknown to me, several of my feeds were not even needed anymore. It would be cool if PackageSourceMapper detected these cases and include the minimal set of <packageSources> entries that are needed.
I did get one hint, like this:

Start creating recommendation nugetPackageSourceMapping.config file.
    - NuGet.org packageSource key matching 523 packages created.
    - nuget-build package source didn't match any packages.
    - dotnet-tools packageSource key matching 10 packages created.
    - azure-sdk-for-net packageSource key matching 13 packages created.
    - NuGetPackageExplorer packageSource key matching 2 packages created.

But in this case my "azure-sdk-for-net" source had all of its packages also on NuGet.org and could therefore be removed.
Ideally it would report that azure-sdk-for-net could be removed since the packages fetched from that feed are the same as those fetched from NuGet.org.
For reference, I did this manual clean-up in Insights so these unnecessary feeds weren't used anymore.
https://github.com/NuGet/Insights/commit/71d09e7534ce2e9e7b9699f443a18dc46b0028a8
Additional Context and Details
No response

@joelverhagen @nkolev92
Currently packagesourcemapper tool works offline, but above one needs internet connection to probe all the sources all the packages. So, it would be quite slow operation. If there's n source m (packageId+version) then we need to make n*m network calls which would quite slow. Is that ok from perf perspective?

It wouldn't be much slower than a restore :)
I'd say it shouldn't be a default, but also I don't think it's worth trying to solve this just yet.


Failing to compile on Windows ARM64

I'm trying to pipx install cibuildwheel[uv] on a Windows ARM64 machine. There are no binary wheels for uv on this platform, so it tries to compile the package from source (uv-0.5.2.tar.gz). However, it is failing with the following error.
         Compiling uv-trampoline-builder v0.0.1 (C:\Users\extra\AppData\Local\Temp\pip-install-hv48jtb7\uv_8840a2c89e304464bb97267224937b3d\crates\uv-trampoline-builder)
      error: couldn't read `crates\uv-trampoline-builder\src\../../uv-trampoline/trampolines/uv-trampoline-aarch64-gui.exe`: The system cannot find the path specified. (os error 3)
        --> crates\uv-trampoline-builder\src\lib.rs:29:5
         |
      29 |     include_bytes!("../../uv-trampoline/trampolines/uv-trampoline-aarch64-gui.exe");
         |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         |
         = note: this error originates in the macro `include_bytes` (in Nightly builds, run with -Z macro-backtrace for more info)

      error: couldn't read `crates\uv-trampoline-builder\src\../../uv-trampoline/trampolines/uv-trampoline-aarch64-console.exe`: The system cannot find the path specified. (os error 3)
        --> crates\uv-trampoline-builder\src\lib.rs:33:5
         |
      33 |     include_bytes!("../../uv-trampoline/trampolines/uv-trampoline-aarch64-console.exe");
         |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         |
         = note: this error originates in the macro `include_bytes` (in Nightly builds, run with -Z macro-backtrace for more info)

      error: could not compile `uv-trampoline-builder` (lib) due to 2 previous errors
      warning: build failed, waiting for other jobs to finish...
      ðŸ’¥ maturin failed
        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit code: 101": `"cargo" "rustc" "--message-format" "json-render-diagnostics" "--manifest-path" "C:\\Users\\extra\\AppData\\Local\\Temp\\pip-install-hv48jtb7\\uv_8840a2c89e304464bb97267224937b3d\\crates\\uv\\Cargo.toml" "--release" "--bin" "uv" "--" "-C" "strip=symbols"`
      Error: command ['maturin', 'pep517', 'build-wheel', '-i', 'C:\\Users\\extra\\pipx\\venvs\\cibuildwheel\\Scripts\\python.exe', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

I don't really know anything about how crates/rust work so I'm not sure how to troubleshoot it. For example, is it sensitive about the unix path seperators (/) in the path? Is it a circular dependency issue where the uv-trampoline crate depends on uv-trampoline-builder but uv-trampoline-builder is using files from the uv-trampoline crate? Is it a version issue where it is depending on a version of the uv-trampoline from before the aarch64 binaries were added? Those are my best guesses.

I think this might be best discussed in https://github.com/astral-sh/uv/issues/1141.

Sure. I wasn't entirely sure if it was actually related to Windows ARM64 specifically or if this was a more general "the sdist package is broken" kind of issue.

Looked into it and I think you're right that there's a real, separate problem with the sdist. Thanks.

If someone could test the "fixed" sdist at https://github.com/astral-sh/uv/actions/runs/11880251223/artifacts/2198442674 that'd be great


fix: bump deps enable strict null checks in test

enable strict null checks for test
bump sfdx-core and oclif/core libraries

Included in a different PR


[React DOM Interactions] Tooltip: Follow Cursor feature

Hello,
I am trying to develop a React Tooltip component that could handle both static tooltip and follow cursor feature (based on a boolean prop follow).
I was trying to develop an extra hook and toggle it based on the top-level prop follow.
  const { getReferenceProps, getFloatingProps } = useInteractions([
    useHover(context),
    useCursorFollow(context, {
      enabled: follow, /*top level boolean prop*/
    }),
  ]);

My approach does not look right since I do not know how to set a virtual element in the reference option from useFloating and also another reference for the DOM element that I have to listen to mouse events.
LE: Not really if this is the right approach but I am taking the DOM reference and as soon as the mouseenter event is triggered I am changing that reference to a virtual element.
Demo here: https://codesandbox.io/s/still-smoke-mxznje?file=/src/App.js

I think there should be some kind of hook that deals with the pointer position. This would be used for both "follow cursor" and context menus' initial position

After experimenting there's definitely a piece missing from the hook that separates the DOM node (which listens for events like hover, etc.) and the position.
In your hook you have to call onOpenChange because calling reference() changes the refs.reference.current value which all the other hooks expect to be a real DOM node, so it never closes after it opens.
To do this properly you need to two separate calls to useFloating(): one that handles the reference DOM node, and the other that handles the virtual reference position.
You could pass the virtual call's reference as a prop to your custom hook and it should then work correctly.
function Tooltip({children, label}) {
  // This handles the DOM node stuff
  const {
    reference: nodeReference,
    floating: nodeFloating,
    strategy,
    context,
    refs,
    update,
  } = useFloating({
    open,
    onOpenChange: setOpen,
  });

  // This handles the virtual position
  const {
    x,
    y,
    reference: virtualReference,
    floating: virtualFloating,
  } = useFloating({
    placement,
    middleware: [offset(5), flip(), shift({padding: 8})],
  });

  const {getReferenceProps, getFloatingProps} = useInteractions([
    useHover(context),
    useFocus(context),
    useRole(context, {role: 'tooltip'}),
    useDismiss(context),
  ]);

  const floatingRef = useMemo(
    () => mergeRefs([nodeFloating, virtualFloating]),
    [nodeFloating, virtualFloating]
  );

  return (
    <>
      {isValidElement(children) &&
        cloneElement(
          children,
          getReferenceProps({
            ref: nodeReference,
            // This is probably better on the document
            // if you have animations
            onMouseMove({clientX, clientY}) {
              virtualReference({
                getBoundingClientRect() {
                  return {
                    width: 0,
                    height: 0,
                    x: clientX,
                    y: clientY,
                    left: clientX,
                    right: clientX,
                    top: clientY,
                    bottom: clientY,
                  };
                },
              });
            },
          })
        )}
      {open && (
        <div
          {...getFloatingProps({
            ref: floatingRef,
            style: {
              position: strategy,
              top: y ?? '',
              left: x ?? '',
            },
          })}
        >
          {label}
        </div>
      )}
    </>
  );
}

Maybe we could come up with something to make this easier without needing to hook calls

This is how I handle the following cursor feature on react-popper. I was hoping that we could make use of useInteractions to avoid duplicating useFloating


Release 0.37.0

Signed-off-by: Patrik Stas<EMAIL_ADDRESS>

Codecov Report

Merging #504 (32fb687) into main (e87eb37) will decrease coverage by 29.27%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main     #504       +/-   ##
===========================================
- Coverage   55.05%   25.77%   -29.28%
===========================================
  Files         154       18      -136
  Lines       14663      908    -13755
  Branches     4094      180     -3914
===========================================
- Hits         8072      234     -7838
+ Misses       2342      579     -1763
+ Partials     4249       95     -4154

Flag
Coverage Δ

integration-aries-vcx
?

unittests-agency-client
25.77% <ø> (ø)

unittests-aries-vcx
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

aries_vcx/src/handlers/issuance/holder.rs

...tation/verifier/states/presentation_request_set.rs

aries_vcx/src/libindy/utils/crypto.rs

...tocols/issuance/issuer/states/proposal_received.rs

aries_vcx/src/utils/mockdata/mock_settings.rs

...c/protocols/connection/invitee/states/responded.rs

aries_vcx/src/libindy/credentials/mod.rs

...cx/src/messages/proof_presentation/presentation.rs

...c/messages/out_of_band/handshake_reuse_accepted.rs

aries_vcx/src/handlers/connection/connection.rs

... and 126 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e87eb37...32fb687. Read the comment docs.


podman 4.4.3

Fixes #19
Fixes #33
Fixes #34
Fixes #35
Fixes #36

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

@conda-forge/podman This is ready for review/merge.

LGTM but since @manics has been more active here I'll give him some time to take a look himself.

@dhirschfeld I would merge this tomorrow, OK? Seems like @manics is currently busy with other things.

Sorry, fell off my radar, merging now!

Sorry, too much other stuff to do! Thanks for fixing and reviewing.


[AutoPR logic/resource-manager] [LogicApps] New Api version

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/6601

This PR has been merged into https://github.com/Azure/azure-sdk-for-go/pull/5811


Textarea

https://www.pivotaltracker.com/story/show/142367873
Text area component.
I cleaned up the existing one, wrote tests and fixed a bug making the component extremely sluggish while resizing.

@Austio should be good now


output accumulation value is very negative

After training for a long time ~81,990 steps on instant ngp, get the following stack trace:

Hi @evonneng could you try the following solution https://github.com/nerfstudio-project/nerfstudio/pull/910 and let me know if this fixes your issue?


MSID: 98280 Version: 1 DOI: 10.1101/2<IP_ADDRESS>8985

MSID: 98280
Version: 1
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>8985
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=98280
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>8985

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/98280v1
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>8985
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/98280/v1/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

Updating author list to include full given-names, as in eJP.


How to solve AttributeError: module 'tensorflow.contrib.learn' has no attribute 'estimators'

File "udc_predict.py", line 54, in
estimator._targets_info = tf.contrib.learn.estimator.tensor_signature.TensorSignature(tf.constant(0, shape=[1,1]))
AttributeError: module 'tensorflow.contrib.learn' has no attribute 'estimator'

i cannot find any useful solution   _ORZ

I did the following change, and some how it worked for me.
first, add this import command in "udc_predict.py"
from tensorflow.contrib.learn.python.learn.estimators import *
change the following line from
estimator._targets_info = tf.contrib.learn.estimators.tensor_signature.TensorSignature(tf.constant(0, shape=[1,1]))
to
estimator._targets_info = tensor_signature.TensorSignature(tf.constant(0, shape=[1,1]))
Also at the bottom, change the print command from
print("{}: {:g}".format(r, prob[0,0]))
to
print("{}: {:g}".format(r, next(prob)[0]))
Hope it helps. I am using Tensorflow Version 1.8, Python 3.6.5

having the similar issue in google colab?
feature_columns = tf.contrib.learn.infer_real_valued_columns_from_input_fn
module 'tensorflow' has no attribute 'contrib'

the error is in function 'create_model_fn' in cdc_model.py,
move "batch_size = targets.get_shape().as_list()[0]" after  "if mode == tf.contrib.learn.ModeKeys.EVAL:" can fix this problem.
the reason is when in PREDICT mode, model_fn deal the param 'targets' as None.


fix(@angular-devkit/build-angular): remove prerelease @angular/compiler-cli peer dependency

Prerelease versions of the @angular/compiler-cli package are not supported and should not be considered valid for a stable version of @angular-devkit/build-angular.
/cc @IgorMinar

@clydin, can you rebase to make all checks green?

It was.  There's an issue with the required check configuration on 8.3.x


[Bug]: MapboxGL.UserLocation heading is not rotating

Mapbox Implementation
Mapbox
Mapbox Version
rnmapbox/maps#main
Platform
Android
@rnmapbox/maps version
rnmapbox/maps#main
Standalone component to reproduce

<MapboxGL.UserLocation
        visible={true}
        showsUserHeadingIndicator={true}
        renderMode={'normal'}
      />

Observed behavior and steps to reproduce

I am using pixel 7 pro.  User location head is not at all rotating with phone direction
Expected behavior
Should rotate along with phone direction
Notes / preliminary analysis
No response
Additional links and references
No response

Same on prod. It was working but now not working.

this is working on 10.0.0-rc.11

useEffect(() => {
MapboxGL.locationManager.start();
return () => {
MapboxGL.locationManager.stop();
};
}, []);
add this and its should be working

I fixed the issue. My token expired. I generated new token it fixed 👍🏼👍🏼

Pls reopen if the issue still happens with 10.1


Fix #428

Related with: https://github.com/CocoaPods/CocoaPods/issues/5400
I tried the solution proposed by @Orta and it works

LGTM

Pushing to trunk now. TYVM.

Thanks!


turn:bus:lanes (special turn rules for buses)

https://wiki.openstreetmap.org/wiki/Key:turn:bus:lanes
It is kind of weird and may be and unique to Kraków, but here bus lanes are often constructed from stretches of

only for special vehicles (buses, taxi, motorcycles, police and ambulances in nonemergency mode etc)
for anyone turning right, and for all that special vehicles for turning right or going straight

See for example https://www.openstreetmap.org/way/217209131 turn:lanes=||right turn:bus:lanes=||through;right
Not sure is it ridiculous to implement or something along "feel free to make a PR"
Right now it is used just 73 times across entire Earth, but ignoring it in Kraków would result in a noticeable worse bus behavior (though it is kind of ridiculous, feel free to WONTFIX it)

There are actually plenty of these in downtown Seattle and along Aurora Ave that I'm familiar with. I don't think I've seen them tagged yet, but I can start adding them.
I don't know how to sanely implement this, so backlog for now, but it is important.

Let me know if https://wiki.openstreetmap.org/wiki/Key:turn:bus:lanes requires some improvements


Archive missing September and October 1978

Dear NYTimes API developers,
The archive API returns zero hits for the months of September and October 1978. Below I paste the calls and responses that I got, which work normally for any other month in the archive:
Call: https://api.nytimes.com/svc/archive/v1/1978/10.json
Response: {"copyright":"Copyright (c) 2013 The New York Times Company.  All Rights Reserved.","response":{"meta":{"hits":0},"docs":[]}}
Call: https://api.nytimes.com/svc/archive/v1/1978/9.json
Response:{"copyright":"Copyright (c) 2013 The New York Times Company.  All Rights Reserved.","response":{"meta":{"hits":0},"docs":[]}}
Is the NYT archive really empty those two months or is there any issue I'm missing?
Best regards,
David Garcia

From Wikipedia
https://en.m.wikipedia.org/wiki/The_New_York_Times
Because of holidays, no editions were printed on November 23, 1851; January
2, 1852; July 4, 1852; January 2, 1853; and January 1, 1854.[179]
https://en.m.wikipedia.org/wiki/The_New_York_Times#cite_note-reprint_disclaimers-181
Because of strikes https://en.m.wikipedia.org/wiki/Strike_action, the
regular edition of The New York Times was not printed during the following
periods:[180]
https://en.m.wikipedia.org/wiki/The_New_York_Times#cite_note-182

December 9, 1962, to March 31, 1963. Only a western edition was
printed because of the 1962–63 New York City newspaper strike
https://en.m.wikipedia.org/wiki/1962–63_New_York_City_newspaper_strike
.
September 17, 1965, to October 10, 1965. An international edition was
printed, and a weekend edition replaced the Saturday and Sunday papers.
August 10, 1978, to November 5, 1978. A multi-union strike shut down
the three major New York City newspapers. No editions of The New York
Times were printed.[179]
https://en.m.wikipedia.org/wiki/The_New_York_Times#cite_note-reprint_disclaimers-181
Two
months into the strike, a parody of The New York Times called Not The
New York Timeswas distributed in the city, with contributors such as Carl
Bernstein https://en.m.wikipedia.org/wiki/Carl_Bernstein, Christopher
Cerf
https://en.m.wikipedia.org/wiki/Christopher_Cerf_(musician_and_television_producer)
, Tony Hendra https://en.m.wikipedia.org/wiki/Tony_Hendra and George
Plimpton https://en.m.wikipedia.org/wiki/George_Plimpton.[181]
https://en.m.wikipedia.org/wiki/The_New_York_Times#cite_note-183[182]
https://en.m.wikipedia.org/wiki/The_New_York_Times#cite_note-184

On Apr 24, 2018, at 4:13 AM, David Garcia<EMAIL_ADDRESS>wrote:
Dear NYTimes API developers,
The archive API returns zero hits for the months of September and October
1978. Below I paste the calls and responses that I got, which work normally
for any other month in the archive:
Call: https://api.nytimes.com/svc/archive/v1/1978/10.json
Response: {"copyright":"Copyright (c) 2013 The New York Times Company. All
Rights Reserved.","response":{"meta":{"hits":0},"docs":[]}}
Call: https://api.nytimes.com/svc/archive/v1/1978/9.json
Response:{"copyright":"Copyright (c) 2013 The New York Times Company. All
Rights Reserved.","response":{"meta":{"hits":0},"docs":[]}}
Is the NYT archive really empty those two months or is there any issue I'm
missing?
Best regards,
David Garcia
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NYTimes/public_api_specs/issues/42, or mute the thread
https://github.com/notifications/unsubscribe-auth/AKjY4ZvgeMtf_X9bto85A7Lr4W05MJq4ks5trt6jgaJpZM4ThMuW
.

That was quick and tremendously useful, thanks a lot!
Best,
David


Allow multiple apps to be installed using a single command

Description of the new feature/enhancement
Let us call winget install with a list of apps, so it'll install them at the same time OR in sequence.
Proposal for command: winget install app-1 app-2 app-3

Or via config file like Chocolatey (e.g. packages.config)

I support it!

This is not the same thing as #156

I support this!

@denelon  @brunovieira97
managing multiple apps in a single command should be supported for all commands ,    not install only but for list / upgrade / uninstall too.

@denelon @brunovieira97
managing multiple apps in a single command should be supported for all commands , not install only but for list / upgrade / uninstall too.

I definitely agree with you! At least uninstall should be covered as well for feature parity between installation and removal process of an app. But implementing this for upgrade too would be very much welcome!

To work around this limitation, I added the following PowerShell function to my profile (notepad $profile):
# Function to process multiple WinGet Upgrades
function Winget-Upgrade {
	Param(
		# List of packages to upgrade
		[Parameter(Position = 0, Mandatory = $true)]
		[string[]] $Packages
	)
	foreach ($Package in $Packages) {
		$Package = $Package.Trim()
		Write-Host ('Upgrading {0}...' -f $Package) -ForegroundColor Green
		WinGet Upgrade $Package
	}
}

With this in place you can upgrade multiple packages like this:
Winget-Upgrade Package1.Package1,Package2.Package1

I also think that this should be supported. It would make usage a lot easier.
I know that you can script it or generate a json file for future use. But it is not nearly as easy as apt-get, chcolatey, yum, etc.

With this in place you can upgrade multiple packages like this: Winget-Upgrade Package1.Package1,Package2.Package2

Excellent.
For others: After adding this to your profile, don’t forget to reload the profile by doing . $profile

This was the first thing I wanted to do, when I just used it for the first time.
@danstis thx for the script.
I have not used power shell scripts util now, so I had to change the execution policy in order to conveniently use winget.

With this in place you can upgrade multiple packages like this: Winget-Upgrade Package1.Package1,Package2.Package2

Excellent.
For others: After adding this to your profile, don’t forget to reload the profile by doing . $profile

There is no such commnd as ". $profile" not at cmd nor at PS. What did you mean ?

Winget-Upgrade

There is not such app as Winget-Upgrade. If you mean Winget Upgrade - than it's still doesn't work.

That should be available without any PS movement!

When and how ill we get the new release ?

When and how ill we get the new release ?

This feature will land in v1.5 which is Due by April 30, 2023
For more: https://github.com/microsoft/winget-cli/milestone/39

This feature will land in v1.5 which is Due by April 30, 2023
For more: https://github.com/microsoft/winget-cli/milestone/39

And if i joined the Windows Package Manager Insider program?

And if I joined the Windows Package Manager Insider program?

I think this is it: https://devblogs.microsoft.com/commandline/windows-package-manager-preview/

I think this is it: https://devblogs.microsoft.com/commandline/windows-package-manager-preview/

Will it appears here ? https://github.com/microsoft/winget-cli/tags
How fast ?

We're looking to cut another preview in the next week or two. It will show up in the releases as a "preview" and it will be published to Windows Insider Dev.

The below winget command do work for winget install-ing multiple packages at once via their ids.
As pointed above by @denelon @brunovieira97 and others, it would also be great to support multiple package ids for uninstall and maybe upgrade commands, for feature parity.
winget install package-id-1 package-id-2 --accept-package-agreements --accept-source-agreements

Indeed, I was using the same arguments for install and uninstall --disable-interactivity --accept-package-agreements --accept-source-agreements although uninstall does not support --accept-package-agreements (which makes sense). Great to hear!


AltHold with LPE broken with latest master

AltHold using LPE with latest master seems not to work properly anymore:
http://logs.uaventure.com/view/ec9QTt8uVfnTMXiatRxdNX#Local_Z_PLOT

I'm not aware of any relevant changes. @jgoppert Can you have a look?

We added an acceleration controller interface - are you using off board control?

That is why I was also very curious - I did not expect any issues here.

@ecmnet what did you havev MPC_ALT_MODE set to. If it is set to 0 you will see local z holding the set point. If you set it to MPC_ALT_MODE = 1 you will see dist bottom being regulated to the set point. I will fly master today to investigate further.

@jgoppert MPC_ALT_MODE was set to 1. Flight was indoors above flat ground with LIDAR Sf10a. I also double-checked Raw LIDAR measurement which is working perfectly, but I some had sonar-timeouts.

@ecmnet So you had good flight on master before with this param setup correct?  So I should look for recent changes in master?

I analyzed your log. It seems very odd to me that the altitude set point is exactly the current altitude except for a couple of times where it is constant. This suggests it must be a recent change in mc_pos_control.  @LorenzMeier any ideas?

@jgoppert Did my last test-flights April, 23 and had no issues up to then for many times.

@ecmnet @LorenzMeier  It really points to this commit then: https://github.com/PX4/Firmware/commit/95faba391a8b6d891409679bc6dd94598b47f06b
This modified the altitude setpoint logic. We just have to figure out what broke.

@ecmnet Are you using off board control or not? If you do, you need to make sure your flags are set up correctly and do not enable acceleration control: control_mode.flag_control_acceleration_enabled = !offboard_control_mode.ignore_acceleration_force &&  		  !status.in_transition_mode;

@LorenzMeier @jgoppert I did not use off board control,  but I will double-check again and let you know.

Is this only present with lpe, or other estimators as well?

@jgoppert I did only check LPE as I am not using other estimators. Also I confirm, that I have not been using off board mode.

@jgoppert Seems to be LPE. Although raw distance is correct, the LPE does not consider LIDAR measurements due to LIDAR beta faults. I cannot remember having those before.

Update: After a beta fault, LIDAR does not recover. Estimation seems to be based on Baro only. Here's a long-time recording (LIDAR vs. LPE-Z):

@ecmnet Long term drift in local z is normal, please plot alt.dist_bottom to see if that is drifting.
The reason local z should drift is that it follows baro, since this is the best measuremnt of global altitude, but the difference between the terrain and the global altitude is measured by the lidar, and that should be constant if the vehicle is stationary and the lidar is reading the same.

@jgoppert Thanks for clarifying this. The difference between global altitude and terrain is constant:

@ecmnet Ok, that looks good, but you still have issues while flying correct? So that must be related to the setpoint issue.

@jgoppert I just tried to reset all params, re-calibrated sensors. Could it be that there was a parameter mess, when flashing the latest master? I'll do another test-flight tomorrow.

After resetting parameters, altHold works again. Closing this issue


Add jenkins extension

Description
Add jenkins extension
Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

@pernielsentikaer this extension can be reviewed, thanks