Running tensorflow distributed on Multiple workers

Click to expand!
Issue Type
Support
Have you reproduced the bug with TF nightly?
Yes
Source
source
Tensorflow Version
2.6
Custom Code
Yes
OS Platform and Distribution
Linux HPC
Mobile device
No response
Python version
3.8.3
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
11.3.1
GPU model and memory
4 RTX 2080 Ti workers each having 8 GPUs
Current Behaviour?
I am trying to run tensorflow distributed training code on multiple worker nodes. I initially tried it using the Mirrored strategy using a single worker with multiple GPUs and the code was working fine and the training process was getting distributed among multiple GPUs. when i have tried it with multiple worker nodes, the training process is actually executing seperately on different workers rather than the load getting distributed among the workers.
My Linux systems has 4 worker nodes with each node having 8 2080Ti GPUs and the GPUs are connected through PCIe system. I also not sure How to configure the TF_config. can anyone help me on this ?

Standalone code to reproduce the issue
if __name__ == "__main__":
    # Get mpi rank
    from getOneHot import getOneHot
    from mpi4py import MPI
    comm = MPI.COMM_WORLD
    rank = comm.Get_rank()

    # Load in the parameter files
    from json import load as loadf
    with open("params.json", 'r') as inFile:
        params = loadf(inFile)

    # Get data files and prep them for the generator
    import tensorflow
    from tensorflow import distribute as D
    callbacks = []
    devices = getDevices()
    print(devices)
    set_tf_config_mpi()
    strat = D.experimental.MultiWorkerMirroredStrategy(
            communication=D.experimental.CollectiveCommunication.NCCL)

# Create network
    from sys import argv
    resume_training = False
    print(argv)
    if "resume_latest" in argv:
        resume_training = True
    with strat.scope():
        # Scheduler
        if isinstance(params["learning_rate"], str):
            # Get the string for the importable function
            lr = params["learning_rate"]
            from tensorflow.keras.callbacks import LearningRateScheduler
            # Use a dummy learning rate
            params["learning_rate"] = 0.1
            # model = create_model(**params)
            # Get the importable function
            lr = lr.split(".")
            baseImport = __import__(lr[0], globals(), locals(), [lr[1]], 0)
            lr = getattr(baseImport, lr[1])
            # Make a schedule
            lr = LearningRateScheduler(lr)
            callbacks.append(lr)
        # Resume Model?
        model_name = None
        if resume_training:
            initial_epoch, model_name = getInitialEpochsAndModelName(rank)
        if model_name is None:
            initial_epoch=0
            model = create_model(**params)
            resume_training = False
        else:
            from tensorflow.keras.models import load_model
            model = load_model(model_name)
    # Load data from disk
    import numpy
    if "root" in params.keys():
        root = params['root']
    else:
        root = "./"
    if "filename" in params.keys():
        filename = params["filename"]
    else:
        filename = "150MeV_all_shuffled_normed.csv"

    restricted = [
            'euc1', 'e1', 'x1', 'y1', 'z1',
            'euc2', 'e2', 'x2', 'y2', 'z2',
            'euc3', 'e3', 'x3', 'y3', 'z3',
            ]
   x, y = getOneHot("{}/{}".format(root, filename), restricted=restricted, **params)
    # val_filename = "150MeV_180kMUmin-stdCC_stitched_triples_dtot_trip_only.csv"
    # val_x, val_y = getOneHot("{}/{}".format(root, val_filename), restricted=restricted)
    val_x, val_y = None, None
    params["gbatch_size"] = params['batch_size'] * len(devices)
    print("x.shape =", x.shape)
    print("y.shape =", y.shape)
    print("epochs  =", params['epochs'], type(params['epochs']))
    print("batch   =", params['batch_size'], type(params['batch_size']))
    print("gbatch  =", params['gbatch_size'], type(params['gbatch_size']))
    # Load data into a distributed dataset
    # Dataset object does nothing in place:
    # https://stackoverflow.com/questions/55645953/shape-of-tensorflow-dataset-data-in-keras-tensorflow-2-0-is-wrong-after-conver
    from tensorflow.data import Dataset
    data = Dataset.from_tensor_slices((x, y))

    # Create validation set
    v = params['validation']
    if val_x is not None:
        vrecord = val_x.shape[0]
        val  = Dataset.from_tensor_slices((val_x, val_y))
        validation = val # data.take(vrecord)
    else:
        vrecord = int(x.shape[0]*v)
        validation = data.take(vrecord)
    validation = validation.batch(params['gbatch_size'])
    validation = validation.repeat(params['epochs'])
    # Validation -- need to do kfold one day
    # This set should NOT be distributed
    vsteps = vrecord // params['gbatch_size']
    if vrecord % params['gbatch_size'] != 0:
        vsteps += 1
    data    = data.batch(params['gbatch_size'])
    data    = data.repeat(params['epochs'])
    records = x.shape[0] # - vrecord
    steps   = records // params['gbatch_size']
    if records % params['gbatch_size']:
        steps += 1
    print("steps   =", steps)
    # Note that if we are resuming that the number of _remaining_ epochs has
    # changed!
    # The number of epochs * steps is the numbers of samples to drop
    print("initial   cardinality = ", data.cardinality())
    print("initial v cardinality = ", data.cardinality())
    data       = data.skip(initial_epoch*steps)
    validation = validation.skip(initial_epoch*vsteps)
    print("final     cardinality = ", data.cardinality())
    print("final v   cardinality = ", data.cardinality())
    # data = strat.experimental_distribute_dataset(data)
    # Split into validation and training
    callbacks  = createCallbacks(params, callbacks, rank, resume_training)
    print(callbacks)
    print("fitting model")
    print(data)

    import tensorflow as tf
    options = tf.data.Options()
    options.experimental_distribute.auto_shard_policy = tf.data.experimental.AutoShardPolicy.OFF
    train_data = data.with_options(options)
    val_data = validation.with_options(options)

    history = model.fit(train_data, epochs=params['epochs'],
            batch_size=params['gbatch_size'],
            steps_per_epoch=steps,
            verbose=0,
            initial_epoch=initial_epoch,
            validation_data=val_data,
            validation_steps=vsteps,
            callbacks=callbacks)
    print("fitting model done")
    if rank == 0:
        model.save("model-final")
    else:
        model.save("checkpoints/model-tmp")

############### slurm script :

#!/bin/bash
#SBATCH --job-name=job1 # Job name
#SBATCH --mem=30000 # Job memory request
#SBATCH --gres=gpu:4 # Number of requested GPU(s)
#SBATCH --time=3-23:00:00 # Time limit days-hrs:min:sec
#SBATCH --constraint=rtx_2080 # Specific hardware constraint
#SBATCH --error=slurm.err # Error file name
#SBATCH --output=slurm.out # Output file name
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --array=1-2%1

if [ -d "model-final" ]
then
scancel $SLURM_ARRAY_JOB_ID
else
module load Anaconda3/2020.07
module load TensorFlow/2.6.0-foss-2021a-CUDA-11.3.1
mpirun python -u main.py resume_latest
fi

Relevant log output
I log each and every epoch in a csv file. I see two csv files created and each one has different workers running.

Hi, @dgscharan
Apologize for the delayed response and you can refer our official documentation for tf.distribute.MultiWorkerMirroredStrategy and TF_CONFIG and distributed training guide which may help you to to configure the TF_CONFIG.
@SuryanarayanaY, Could you please look into this issue ? Thank you!

Hi @dgscharan ,
For multi-worker training, as mentioned before, you need to set up the 'TF_CONFIG' environment variable for each binary running in your cluster. For more details please refer to the source here.
Please also refer to this tutorial on how to setup multi-workers with custom training.


emmet grouping not working

VSCode Version:    1.13.1

OS Version:  Win 10 Simp Chinese
useNewEmmet is set to true

step to reproduce:
open a new html and input   (div>dl>(dt+dd)*3)+footer>p

expect result:
expand to

actual result:
only  div>dl  remains，  dt and dd are gone.

extra info:
revert to 1.12 fix this problem

This is being tracked in #27784
You dont need to revert to 1.12 to fix the problem.
You can just set emmet.useNewEmmet to false


RangeSlider-Added DisableMin and DisableMax

Tested using new parameters in docs example code

Thanks, do you want to add Readonly also?

Thanks, do you want to add Readonly also?

I tried really fast to do the same as for the disable (but obviously replacing it for "readonly" on the backend "input") but it wasn't working! I am not sure why


Indentation error

pylint is giving me an indentation error for almost all of my code. But when I fixed it on google colab, colab gave an unexpected indentation error
My code works well, I am not sure how useful linting is

@AhmedCode99 Can you share your code over here?

pylint is giving me an indentation error for almost all of my code. But when I fixed it on google colab, colab gave an unexpected indentation error
My code works well, I am not sure how useful linting is

Are you linting the file .ipynb file with pylint? If so, it is natural to throw out error. Pylint only works for .py file.


Let's Workshop! - Introducing December's NEWEST MUDDLErs

@JonHolt
Thank you so much for forking us! I want to introduce you to MUDDLE's Technical Editor, Rebecca @RJP43. She will be your number one resource for navigating the technical aspects of the project. If you have any questions before you get MUDDLING just let us know! So glad to have you :smiley:
Best, Tay

@gmoe
Thank you so much for forking us! As you are already familiar with the team and a CS master, I'll spare you a formal introduction :) So so so glad to have you on board! We'll meet ya in the
backyard! :house_with_garden: :smiley:
Best, Tay


Not using shallow copy if mapping rule exists

This PR fixes #628
Added checking if a mapping conversion is defined for desired type

Thank you!


Generics and Fns: I'm doing them wrong

Throw away the current implementation. I did it to solve the problem of type-checking functions at runtime, hence the need for an anchor which had a concrete type. It turns out this is BS: we shouldn't do it. Function type checking is something for a type-checked language. Metaty should stick to records and generic records.
The basic API is something like:
-- A table that has a default key for it's get method
local DT = record'DefaultTable'
  :generic('K', Any)
  :generic('A', Any)
  :field('defaultKey', g'K')
  :field('data', Table{K=g'K', V=g'A'}) -- note: changed variable name
  :done()

DT.__index = function(self, k)
  return self.data[k or self.defaultKey]
end

-- Note: further calls to the same types will return the same instance
local T_SN = Table{K='string', V='number'}
local DT_SN = DefaultTable{K='string', A='number'}

-- passes type checking
local d1 = DT_SN{defaultKey='default', data=T_SN{default=4, other=7}}

-- will fail type checking on BOTH the DT defaultKey and the Table
local d2 = DT_SN{defaultKey=4, data=T_SN{default='foo', other='bar'}}

Basic approach
When a generic is instantiated (i.e. local T_SN = Table{K='string', V='number'}) it sets a "source of truth" for the name mapping and goes through all the fields of the generic to replace the types.
The resulting type itself can still be generic if any of the genvars are still generic, i.e.
  :field('data', Table{K=g'K', V=g'A'}) -- note: changed variable name

The resulting table type is still generic except the constraint on the table's V is now g'A' instead of it's previous value of Any. We also need to check that the constraints on the parent type were valid for our constraints (probably need to pass the constraints through or something).
The point is that at the end we have a concrete type if none of the params were generic or a generic type if any of them were generic. The concrete type will have a concrete constructor, whereas the generic type will still have a generic constructor.

No, no generics. #2


feat(stats): split off transmission to RawSender, implement batching and queueing support, add streamlined prefix and suffix support

Motivation
This pull request achieves the goal originally set out in dash#5167, to migrate the base of our Statsd client implementation to one that is actively maintained. Statoshi (source) utilizes talebook/statsd-client-cpp, which in turn is inherited by Dash.
As Statsd is the only cross-platform reporting mechanism available (USDT requires a Linux host with superuser privileges and RPCs don't provide as much flexibility as desired), emphasis is placed on using Statsd as the primary way for the Dash daemon to report metrics related to node and network health.
As part of maintaining our Statsd client, this PR aims to migrate the base of our implementation to vthiery/cpp-statsd-client, a streamlined implementation that embraces C++11 with a thread-safe implementation of queueing and batching, which reduces the number of packets used to transmit stats.
These capabilities are optional and users can opt not to use them by setting -statsduration=0 to disable queueing (which will also disable batching) or -statsbatchsize=0, which will disable batching (but will not disable queueing unless requested explicitly).
Additional Information

Dependent on https://github.com/dashpay/dash/pull/5167
RawSender (and by extension, RawMessage) strive to remain as unopinionated as possible, moving the responsibility to construct valid Statsd messages onto StatsdClient. This is to ensure that  RawSender can be reused down the line or independently extended without impacting the Statsd client.

RawMessage exists to provide extensions to std::vector<uint8_t> that make it easier to abstract away strings while also implementing some of its semantics like append() (and its alias, +=).

InitStatsClient() was introduced to keep StatsdClient indifferent to how arguments are obtained and sanitized before they're supplied to the constructor. This is to keep it indifferent to all the backwards-compatibility code for deprecated arguments still work.
When constructing the Statsd message, we can use %f without having to specify a precision as tinyformat automatically assumes a precision of 6 (source) and problems don't seem to be observed when using %f with integers (source).

As a guardrail, there is a static_assert to ensure that a specialization of send() involving a non-arithmetic type will raise alarm when compiling (source).

Breaking changes

-statsenabled (replaced with specifying -statshost), -statshostname (replaced by -statssuffix) and -statsns (replaced by -statsprefix) have been deprecated and will be removed in a future release.

Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have added or updated relevant unit/integration/functional/e2e tests (note: N/A)
[x] I have made corresponding changes to the documentation
[x] I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

Should update test/util/data/non-backported.txt
diff --git a/test/util/data/non-backported.txt b/test/util/data/non-backported.txt
index 150bbb70c1..7684074874 100644
--- a/test/util/data/non-backported.txt
+++ b/test/util/data/non-backported.txt
@@ -29,7 +29,8 @@ src/rpc/quorums.cpp
 src/saltedhasher.*
 src/spork.*
 src/stacktraces.*
-src/statsd_client.*
+src/stats/*.cpp
+src/stats/*.h
 src/test/block_reward_reallocation_tests.cpp
 src/test/bls_tests.cpp
 src/test/dip0020opcodes_tests.cpp

not sure if can actually do https://github.com/dashpay/dash/pull/6267/commits/d42c88fa33d0763843edc4d073e24e785eac5de9
develop:
network.hashesPerSecond:103824.436286|g
network.terahashesPerSecond:0.000000|g
network.petahashesPerSecond:0.000000|g
network.exahashesPerSecond:0.000000|g
network.difficulty:0.003205|g
this PR:
network.hashesPerSecond:103185|g
network.terahashesPerSecond:1.03185e-07|g
network.petahashesPerSecond:1.03185e-10|g
network.exahashesPerSecond:1.03185e-13|g
network.difficulty:0.00335768|g

could probably make it work with smth like
diff --git a/src/stats/client.cpp b/src/stats/client.cpp
index cdcf36d281..c2a1f4854d 100644
--- a/src/stats/client.cpp
+++ b/src/stats/client.cpp
@@ -158,6 +158,7 @@ bool StatsdClient::send(const std::string& key, T1 value, const std::string& typ
     }

     // Construct the message and if our message isn't always-send, report the sample rate
+    if (value < EPSILON) value = 0;
     std::string buf{strprintf("%s%s%s:%s|%s", m_prefix, key, m_suffix, ToString(value), type)};
     if (frequency < 1.f) {
         buf +=<EMAIL_ADDRESS>frequency);

does NOT LGTM yet cc1a75a
, but I have no objections to get it merged now

what does this mean? Do you approve or not?

what does this mean? Do you approve or not?

edited message to make it more clear.


Create deletion_chooser.lua

A re-imagining of delete_selective.lua.
I wanted to add new items to delete_selective.lua but it grew up to 20 menu items and that's too many even for me. This is an experiment using a single-menu script to present a macro-palette dialog to choose the 20 items, with configurable "code" keys, mimicking Keyboard Maestro's palettes (cf: @Nick-Mazuk). Call the script, type the key, hit Return.
These are the items that I most commonly like to delete independently. Obviously the list is as long as a piece of string but this feels like a reasonable compromise - inclusivity vs prolixity.

Interesting to note that while entry ReverseStem flags are present FCPrimaryBeamMods are at least partially masked and won't "delete".

It would be nice if the keystroke itself closed the window

Looked as hard as I could for a solution with no luck.


[Feature request] Manage "Do not disturb" in macOS

Pomotroid should manage the "Do not disturb" setting in macOS as follows:

When a pomodoro starts, switch "Do not disturb" on
When a break starts, switch "Do not disturb" off

This helps with focussing on the task during the pomodoro duration and answering instant messages during the 5 minute break.
This should be an optional feature that the user can turn off in settings.

I also would like this....
There seems to be no API, but some workaround are possible.


[Bug] ENAMETOOLONG on "preset-create-react-app@4" package installation

Describe the bug
Since @storybook/preset-create-react-app v4, I can no longer install this package.
For this to work, I have to manually insert "<EMAIL_ADDRESS>into yarn.lock and replace "@storybook/react-docgen-typescript-plugin" canary with "@storybook/react-docgen-typescript-plugin" "1.0.2-canary.253f8c1.0"
Steps to reproduce the behavior
yarn add @storybook/preset-create-react-app -D

Screenshots and/or logs
error An unexpected error occurred: "ENAMETOOLONG: name too long, unlink '/home/abera/.cache/yarn/v6/npm-@storybook-react-docgen-typescript-plugin-1.0.2-canary.6.9d540b91e815f8fc2f8829189deb00553559ff63.0-3103532ff494fb7dc3cf835f10740ecf6a26c0f9-integrity/node_modules/@storybook/react-docgen-typescript-plugin'".

Expected behavior
Change version of "@storybook/react-docgen-typescript-plugin" from "canary" to "1.0.2-canary.253f8c1.0", or set this dependency as peer dependency (it is already included in "@storybook/react")
Environment

OS: Linux Mint
Node.js version: 14.19.1
Yarn version: 1.22.17

Problem solved by : storybookjs/storybook/discussions/18441
Seems to be caused by encryptfs. Just need to set the cache folder in the root directory.


Provide control over the source of the timestamp

This allows selecting between getting the timestamp from the camera or from the system running the IOC.
The system timestamp is obtained from the aravis buffer which was saved on the reception of the first packet of the image, this is more precise than timestamping after receiving the full frame.
The reason why this change is needed is because we have certain cameras in which PTP doesn't work and using the system
timestamp in the aravis buffer would be good enough for our application.

Tom suggested renaming the record so that it matches ADVimba's, so I've amended the commit

Given that my use-case turned out to work fine using the epics timestamp, I'm closing the PR


Advice - Add S to Recommandation like in V1X

Mm could you provide details on this one ? The translation is already with a s in my locales files.

OK good then.
I saw the commit in V1.X but not in master and it has no S in our actual GTE apk so it was just a reminder.

It was fixed in 364551015a9c0c6e1162c2b19ee8f1025d24c778


Use built-in flask logging

Rather than 'print' statements

See https://flask.palletsprojects.com/en/2.0.x/logging/


Map tiles seem to be not updating, at least in some instances

Bug description
As noted on Discord by @Rioscac, map tiles seem to be not updating, at least in some places.
We can look at the production vector tiles here:
https://vtiles.openhistoricalmap.org/#18/22.281293/114.159284

Compare to Map Data here:
https://www.openhistoricalmap.org/way/198564637#map=19/22.28114/114.15952&layers=OD&date=1914&daterange=1800,2022

And we see that the map data has more detail in building corners, and in the grassy area around the plaza, which we are not seeing in the vector tiles even many hours after the edits were made.
We'd expect vector tiles to refreshing within a few minutes.

Hi there, map tiles are also not updating in Melbourne Australia.  I made some substantial changes to the Hobsons Bay area of Melbourne, including coastlines, but am also not getting updates from non-coastline elements.  I think most of these changes would've been posted in Changeset: 34691 or after.
I added bridge pathlines to Queensbridge in the CBD (in addition to the bridge area polygon that I drew in a few weeks ago.  These are showing when you go to the edit screen, but are not rendering on the map:

Similarly as part of my coastline updates I corrected the geometry of the Williamstown pier, this is reflecting correctly when using the editor:

However the tiles are still showing the older incorrect geometry:

A few weeks ago I added a large sand polygon over Port Melbourne, as old maps show that is what the area was before European settlement.  I incorrectly put the end date as 1951, the area would've been developed much earlier than that. So I removed the polygon entirely until I have more information about it.  The map tiles are still showing the sand area...

However, you can see when editing (also in JOSM) the large sand area has been removed:

Thanks for the report. Looking into it - this is perhaps related to the last deploy, but still a bit uncertain what's causing this. Need to kick the tile server somewhere. cc @Rub21 @geohacker

@Rub21 @geohacker - the imposm logs look fine, so seems like the tiler-server is updating data correctly. My hunch would be that something is failing with the cache cleaning process, and tiles are not being correctly expired from cache.

Thank you for triaging @batpad! I looked around a bit and could see that replication job had stopped creating new minutely on May 3. It had this error in the logs:
May 03, 2022 10:13:42 PM org.springframework.transaction.support.TransactionTemplate rollbackOnException
SEVERE: Application exception overridden by rollback exception
org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [CREATE TEMPORARY TABLE tmp_nodes ON COMMIT DROP AS SELECT node_id, version FROM nodes WHERE (((xid_to_int4(xmin) >= ? AND xid_to_int4(xmin) <= ?))) AND redaction_id IS NULL]; FATAL: terminating connection due to administrator command; nested exception is org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command

I think this lines up with the deploy? Though I've no clear explanation of why the replication would break on a deploy. We should try to figure this out.
For now, I've just restarted that pod and everything is running. Replication and Tiler has ~2 days worth of data to catchup but should be done shortly.

I can confirm the tiles are now updating.  Thanks very much for your help!


Dispositions will hang indefinitely when unsolicitedly terminating with batching enabled

The following code does not handle the case where the disposition buffer is full but the disposition loop is shut down and will hang indefinitely if the condition is
func (r *Receiver) messageDisposition(ctx context.Context, msg *Message, state encoding.DeliveryState) error {
	var wait chan error
	if r.link.ReceiverSettleMode != nil && *r.link.ReceiverSettleMode == ModeSecond {
		log.Debug(3, "RX (messageDisposition): add %d to inflight", msg.deliveryID)
		wait = r.inFlight.add(msg.deliveryID)
	}

	if r.batching {
		r.dispositions <- messageDisposition{id: msg.deliveryID, state: state}
	} else {

Instead, the function should be selecting on this condition or case <-r.link.Detached: in order to return when the disposition buffer will no longer be drained.

@jhendrixMSFT is there any update on a 0.18 release tag date?


Invalid Copilot token: missing token: 403 and Cannot Login to Copilot in VS Code

VS Code Version: 1.89.1
OS Version:  Windows 11 Home 23H2
Logs - See below

I'm getting the errors in the subject of this post.   Both VS Code and Copilot Chat are updated to the latest versions. I cannot login to Copilot Chat manually either. when I click on the "Chat" icon in the left pane, there is a link to sign-in but when I click on the link one of two things will happen. Either nothing happens at all, or a web page opens with a 502 error, "Web server received an invalid response while acting as a gateway or proxy server." I tried updating Copilot Chat to the pre-release version but it's still happening. Here is a snippet from the log:
2024-05-10 11:14:51.939 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:14:51.940 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:14:51.941 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:14:51.941 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:14:51.943 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:14:51.943 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:14:51.945 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:14:51.952 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:14:51.954 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:14:51.954 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:14:51.956 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:15:57.061 [info] [FetcherService] Using Helix fetcher
2024-05-10 11:15:57.062 [info] [gitExtensionService] Initializing Git extension service.
2024-05-10 11:15:57.437 [info] [gitExtensionService] Successfully activated the vscode.git extension.
2024-05-10 11:15:57.437 [info] [gitExtensionService] Enablement state of the vscode.git extension: true.
2024-05-10 11:15:57.437 [info] [gitExtensionService] Successfully registered Git commit message provider.
2024-05-10 11:15:59.156 [info] [auth] Logged in as AdrianeNS
2024-05-10 11:16:03.217 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:16:03.220 [info] [extension] GitHub Copilot could not connect to server. Extension activation failed: "Failed to get copilot token"
2024-05-10 11:16:03.248 [info] [auth] Logged in as AdrianeNS
2024-05-10 11:16:03.410 [info] [auth] Invalid copilot token: missing token: 403
2024-05-10 11:16:04.440 [info] [RemoteAgents] Invalid remote agent response: unauthorized: not licensed to use Copilot
(SyntaxError: Unexpected token 'u', "unauthoriz"... is not valid JSON)

Do you used copilot-gpt4-service before？

I have same issue,

I have same issue,
2024-05-23 15:45:21.816 [error] [auth] Extension activation failed: "Contact Support. You are currently logged in as leovingo."
2024-05-23 15:45:21.818 [info] [auth] Invalid copilot token: missing token: 403
I have edu account and have use copilot in vsc before

I have same issue, 2024-05-23 15:45:21.816 [error] [auth] Extension activation failed: "Contact Support. You are currently logged in as leovingo." 2024-05-23 15:45:21.818 [info] [auth] Invalid copilot token: missing token: 403 I have edu account and have use copilot in vsc before

I have the same question,and my school is USTC in China.

Do you used copilot-gpt4-service before？

我用了一天，之后就用不了了，我在官网copilot点击start free trial也没有反应，你解决了吗？

Do you used copilot-gpt4-service before？

我用了一天，之后就用不了了，我在官网copilot点击start free trial也没有反应，你解决了吗？

你被封号了，去开工单申诉吧

403 2024-05-27 08:43:34.439 [info] [auth] Invalid copilot token: missing token: 403
Redirected to Github Copilot subscription page to signup for monthly or yearly Copilot account.
Please remove this binding associated with VScode.

403 2024-05-27 08:43:34.439 [信息] [身份验证] 副驾驶令牌无效：缺少令牌：403
重定向至 Github Copilot 订阅页面以注册按月或按年订阅的 Copilot 帐户。 请删除与 VScode 关联的此绑定。

I have get the education plan which copilot is free,but now I can't use it.

403 2024-05-27 08:43:34.439 [信息] [身份验证] 副驾驶令牌无效：缺少令牌：403
重定向至 Github Copilot 订阅页面以注册按月或按年订阅的 Copilot 帐户。 请删除与 VScode 关联的此绑定。

I have get the education plan which copilot is free,but now I can't use i

403 2024-05-27 08:43:34.439 [信息] [身份验证] 副驾驶令牌无效：缺少令牌：403
重定向至 Github Copilot 订阅页面以注册按月或按年订阅的 Copilot 帐户。 请删除与 VScode 关联的此绑定。

I have get the education plan which copilot is free,but now I can't use it.

我也是UCAS的学生包，你是不是存在滥用行为导致被检测了，一般出现403就是被封号了

Hi everyone,
This seems to have slipped through our issue triage scripts and we do apologize for that / the long reply time.
Are you still experiencing the same issue. Can you please try the latest VS Code insiders + Copilot Chat pre-release.

Hi everyone,
This seems to have slipped through our issue triage scripts and we do apologize for that / the long reply time.
Are you still experiencing the same issue. Can you please try the latest VS Code insiders + Copilot Chat pre-release.

Yes we are

@iubayb Can you please provide a screenshot of the chat panel + your Copilot Chat Logs (CTRL / CMD + SHIFT + U -> GitHub Copilot Chat) on the latest VS Code insiders + Copilot Chat pre-release

2024-07-18 17:31:59.544 [info] [auth] Logged in as iubayb
2024-07-18 17:32:00.065 [info] [chat] copilot token chat_enabled: false
2024-07-18 17:32:00.065 [info] [auth] Got Copilot token for iubayb
2024-07-18 17:32:00.094 [info] [auth] Logged in as iubayb
2024-07-18 17:32:00.107 [info] [chat] copilot token chat_enabled: false
2024-07-18 17:32:00.108 [info] [auth] Got Copilot token for iubayb
2024-07-18 17:32:00.844 [info] [ModelMetadataFetcher] Failed to fetch models, SyntaxError: Unexpected token 'u', "unauthoriz"... is not valid JSON
at JSON.parse ()
at t.json (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:88:17938)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at UA._fetchModels (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:1291:3447)
at UA.getChatModel (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:1291:2329)
at ZT.getChatEndpoint (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:1291:5862)
at t.detectIntent (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:840:611)
at JTe (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:845:795)
at dp.fetchResponse (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:848:2336)
at dp.provideInteractiveEditorResponse (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:847:2016)
at Nc.handleCommonInlineChatRequest (/Users/ayoub/.vscode/extensions/github.copilot-chat-0.17.1/dist/extension.js:882:3894)
at g.$invokeAgent (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:154:50217)

@iubayb That's not the latest VS Code Insiders + pre-release. But here I see that Copilot Chat is disabled by your organization
2024-07-18 17:32:00.094 [info] [auth] Logged in as iubayb
2024-07-18 17:32:00.107 [info] [chat] copilot token chat_enabled: false
2024-07-18 17:32:00.108 [info] [auth] Got Copilot token for iubayb

So therefore chat will not work until your organization admin enables it

I tried downloading the latest version of vsc but it still doesn't work


Core estimator occassionally hangs on Firefox with an Intel Core i7 5960X.

Running the core estimator on current Firefox and an Intel Core i7 5960X that has 8 hardware cores and 16 logical cores, core estimator hangs if it finds 16 cores, and then attempts to detect 32, most likely because it tries to spawn 32 simultaneous workers, which never start up. See https://bugzilla.mozilla.org/show_bug.cgi?id=1052398. If I lift the pref dom.workers.maxPerDomain to 40, then the hang is avoided.
(However most of the time, the core estimator does not get near the correct number on this cpu. Running in a sequence on Firefox, I'm getting 7,1,5,1,3,2,1,2,16,1,3,8,16)

Good catch, I hadn't realized there could be a limit. I don't have any (good) ideas at the moment to fix this since the web worker limit isn't available to us in the API.
Idea: Maybe for each worker that is spawned, it can start with posting a message saying that we are started. Then we can have a timeout for each worker checking for the message. If we exceed the timeout, we can stop the script and go with the current estimate.

In the meantime we can hardcode in some ugly UA sniffing code and if we detect Firefox, only ever test up to 20 threads.
If core estimator estimates exactly 20 cores for Firefox, maybe we should add a passive aggressive alert("Your browser is throttling this application. Please uninstall Firefox and install Google Chrome or Microsoft Edge").

Hmm, testing on IE11 today on the 8 hw/16 logical core i7-5960X, I see that IE11 too hangs when running the core estimator live demo page, at the same point where Firefox does: spawning 16 workers was ok, and 32 workers never finishes.

Perhaps it would be possible to have the workers ping back to their creator once they start up. That way if there's no reply in fixed k seconds, the estimator could assume that that many workers is not supported, and the test would run only up to that number of threads. I agree it's ugly, but at least it's a pragmatic workaround for browsers that silently quota the number of workers.

FWIW, IE11 and Edge seem to limit to 25 workers per window or per domain (not sure which one).
I did some testing in https://bugzilla.mozilla.org/show_bug.cgi?id=1241485#c0

I've been trying this with IE 11 on Win 8.1 and ran into it never finishing on the demo page a few times.  Other times it started up quite a few cores and then dialled back to 8 or 4.  I have an u7-4700MQ @ 2.4Ghz with 4 cores (8 with HT).  I suspect this worker limit would explain the problem.
Could there be an option added to the API where we are happy to accept an upper limit on the core count?  With what I want to do - detect barcodes - I'm very happy to use <= 8 cores anyway.  On a low-end tablet I'd want to throttle that back to 1 or 2 cores as required and on a higher-end tablet, like a Surface Pro, I'll use as much as I can up to that reasonable limit.


ServiceLoader prevents jetcd from working in Karaf

I'm trying to get jetcd working in a poject that runs in Karaf. I'm currently working primarily with the 0.0.1 build and I'm hoping somebody can help me out. Basically, in Karaf, the jetcd client can't resolve any hostnames. This is due to the call to ServiceLoader.load(URIResolver.class) in SmartNameResolver.java. My understanding is that it is possible to work around this in karaf by modifying the MANIFEST.MF file in the jetcd-core-0.0.1.jar file. The goal is to tell Karaf that it needs to use the SPI-Fly module to dynamically load the URIResolver implementations by temporarily using a classloader in a different context.
I've made the following change to the manifest:
Require-Capability:osgi.serviceloader;filter:="(osgi.serviceloader=com.coreos.jetcd.resolver.URIResolver)";resolution:=optional;cardinality:=multiple,osgi.extender;filter:="(osgi.extender=osgi.serviceloader.processor)"osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
I've also tried this to see if I could get it to use just the Direct name resolver.
Require-Capability:osgi.serviceloader;filter:="(osgi.serviceloader=com.coreos.jetcd.resolver.URIResolvers$Direct)";resolution:=optional;cardinality:=multiple,osgi.extender;filter:="(osgi.extender=osgi.serviceloader.processor)"osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
So far I've had no luck and a continue to get the same error in karaf:
2017-12-06 11:09:42,611 | WARN  | fault-executor-0 | SmartNameResolver                | 56 - com.foo.jetcd.example - 1.0.0.SNAPSHOT | Error wile getting list of servers
com.coreos.jetcd.exception.EtcdException: Unable to resolve endpoints [http://<IP_ADDRESS>:2379]
at com.coreos.jetcd.exception.EtcdExceptionFactory.newEtcdException(EtcdExceptionFactory.java:30)
at com.coreos.jetcd.resolver.SmartNameResolver.doResolve(SmartNameResolver.java:135)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_151]
at java.lang.Thread.run(Thread.java:748)[:1.8.0_151]
2017-12-06 11:09:42,614 | WARN  | fault-executor-0 | ManagedChannelImpl               | 56 - com.foo.jetcd.example - 1.0.0.SNAPSHOT | [io.grpc.internal.ManagedChannelImpl-1] Failed to resolve name. status=Status{code=NOT_FOUND, description=null, cause=null}
2017-12-06 11:09:42,617 | ERROR | nsole user karaf | example                          | 56 - com.foo.jetcd.example - 1.0.0.SNAPSHOT | [com.foo.jetcd.example.JetcdClient(1)] The start method has thrown an exception
java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: NOT_FOUND
I'm wondering if anyone has been able to get around this issue. Also, if so, I'd like to suggest updating the configuration of the maven-bundle-plugin so that this works out of the box. If somebody can help me figure out what the manifest should look like I'd be happy to submit a PR with the changes to the maven-bundle-plugin configuration.

The latests source should not expose this problem at least the direct resolver is available out of the box. I will work on OSGi stuffs as soon as I have time but could you please check the latest code does not have the same problem ?

Sure, I'll try building from the latest source and let you know what I run into. I know for this version we also had to work around a ClassNotFoundError because of a split package issue between grpc-context and grpc-core, but that may not be something you have a lot of control over.

@lburgazzoli I think I've managed to get it working by building the latest jetcd-core-0.1.0-SNAPSHOT.jar, however, it's somewhat based on trial and error with the other dependencies we've included. In order to get it to install and run properly, we have the following dependencies declared in the pom.xml in our project:
<dependencies>
    <!--jetcd-->
    <dependency>
        <groupId>com.coreos</groupId>
        <artifactId>jetcd-core</artifactId>
        <version>0.1.0-SNAPSHOT</version>
    </dependency>
    <!-- stuff neede to get jetcd to work -->
    <dependency>
        <groupId>com.google.protobuf.nano</groupId>
        <artifactId>protobuf-javanano</artifactId>
        <version>3.1.0</version>
    </dependency>
    <dependency>
        <groupId>com.jcraft</groupId>
        <artifactId>jzlib</artifactId>
        <version>1.1.3</version>
    </dependency>
    <dependency>
        <groupId>com.ning</groupId>
        <artifactId>compress-lzf</artifactId>
        <version>0.9.9</version>
    </dependency>
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-tcnative</artifactId>
        <version>2.0.0.Final</version>
    </dependency>
    <dependency>
        <groupId>com.github.jponge</groupId>
        <artifactId>lzma-java</artifactId>
        <version>1.3</version>
    </dependency>
    <dependency>
        <groupId>net.jpountz.lz4</groupId>
        <artifactId>lz4</artifactId>
        <version>1.3.0</version>
    </dependency>
    <dependency>
        <groupId>org.bouncycastle</groupId>
        <artifactId>bcprov-jdk16</artifactId>
        <version>1.46</version>
    </dependency>
    <dependency>
        <groupId>org.eclipse.jetty.alpn</groupId>
        <artifactId>alpn-api</artifactId>
        <version>1.1.3.v20160715</version>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.7.25</version>
        <scope>provided</scope>
    </dependency>

    <!-- osgi stuff -->
    <dependency>
        <groupId>org.osgi</groupId>
        <artifactId>org.osgi.service.component</artifactId>
        <version>1.3.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.osgi</groupId>
        <artifactId>org.osgi.core</artifactId>
        <version>6.0.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.osgi</groupId>
        <artifactId>org.osgi.service.cm</artifactId>
        <version>1.5.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.osgi</groupId>
        <artifactId>org.osgi.service.component.annotations</artifactId>
        <version>1.3.0</version>
        <scope>provided</scope>
    </dependency>
</dependencies>

You wouldn't be able to advise on a simpler / more correct set of dependencies that would accomplish the same goal, would you?

I'll try to provide a karaf featurex xml for the next release but I'm quite busy so I do not have an ETA so if you have any spare time to work on that I'd more than happy to review and merge a PR.

Understood. It looks like the latest snapshot is the path forward for us, so If I can get something working cleanly I'll submit a PR.


fix: allow custom wallet configuration in wizard flow [FTL-9023]

:tada: This PR is included in version 1.61.2 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


Add project changelog

In order to help all users track changes to this boilerplate project, we’re adding a changelog to the repository.
All notable changes to the project should be documented here 😄

🦀 Requesting reviewers for this pull request:

@garno (contributor with 0 commits in the last 90 days and 2 commits overall)


Add BYO fees to XCM transfer

This has been de-prioritised.

Closing this—unlikely to be done at any point soon, so no need to have a issue in the backlog. We'll create a new ticket (with requirements) if and when we decide to do this.


Document LVM support for storage quotas

LVM backend supports storage quotas while the documentation still refers
to this feature as being unsupported:
ac1740d7092bf0dfb5e70cc66330e60b68e95613
f88e381d2d7c63c3a35f82691cd3627bf029c2b8
https://github.com/lxc/lxd/issues/1205
Signed-off-by: Dmitrii Shcherbakov<EMAIL_ADDRESS>

This pull request didn't trigger Jenkins as its author isn't in the whitelist.
An organization member must perform one of the following:

To have this branch tested by Jenkins, use the "ok to test" command.
To have a one time test done, use the "test this please" command.

Those commands are simple Github comments of the format: "jenkins: COMMAND"


graphic extendshape event

when create a new shape,the methods i found are shape and buildpath,how we get the event and override, so it has default action when used as  graphics component.

Where do you wish to use it? Can you give a demo?

i get another idea access it,use a warpper function  for graphic, i can  drag control points to pan
and scale,thanks


Support Generation for Stress Reinforcement

Hey,
I would like to print a part that is 100% optimized for stress reinforcement (no support free and surface quality aspects). Therefore, often it is needed to have support structures printed. Is the process of support generation implemented in this project? And if not, what is your workflow on using support structures? I found Support_Generation_for_Curved_RoboFDM, but since there is no documentation for it, I don't really know how to use it.
Thank you for the project and your help!

Please refer to the updated version, the ReadMe file is updated.

Thank you for the updated ReadMe over there!


Fix block announcement network encoding

I noticed the following issues reviewing the network encoding of block announcements:

The block-announces substream handshake is missing the best block hash (kagome vs. substrate). This probably also means that the genesis hash is never checked, because it currently contains the best block hash instead.

The BlockAnnounce messages is missing the best block byte, see host spec v0.1.0-beta1 chapter 4.8.1

@FlorianFranzen, first issue is resolved in PR #787. Second one in not actual - internally that coding in according to spec.
This is happening here.


I am getting the following errors when trying to install airpal

Hello,
I am getting the following issues when installing the airpal.
[ec2-user@ip-172-31-37-150 airpal]$ sudo ./gradlew -Dairpal.prestoVersion=0.85 clean shadowJar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-client/0.85/presto-client-0.85.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-root/0.85/presto-root-0.85.pom
Download https://repo1.maven.org/maven2/io/airlift/airbase/28/airbase-28.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-spi/0.85/presto-spi-0.85.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-main/0.85/presto-main-0.85.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.85/presto-parser-0.85.pom
Download https://repo1.maven.org/maven2/io/airlift/http-client/0.97/http-client-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/airlift/0.97/airlift-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/json/0.97/json-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/bootstrap/0.97/bootstrap-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/concurrent/0.97/concurrent-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/node/0.97/node-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/configuration/0.97/configuration-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/discovery/0.97/discovery-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/event/0.97/event-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/floatingdecimal/0.2/floatingdecimal-0.2.pom
Download https://repo1.maven.org/maven2/io/airlift/http-server/0.97/http-server-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/jaxrs/0.97/jaxrs-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/jmx/0.97/jmx-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/jmx-http/0.97/jmx-http-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/log/0.97/log-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/log-manager/0.97/log-manager-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/stats/0.97/stats-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/trace-token/0.97/trace-token-0.97.pom
Download https://repo1.maven.org/maven2/io/airlift/units/0.97/units-0.97.pom
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-client/0.85/presto-client-0.85.jar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-spi/0.85/presto-spi-0.85.jar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-main/0.85/presto-main-0.85.jar
Download https://repo1.maven.org/maven2/com/facebook/presto/presto-parser/0.85/presto-parser-0.85.jar
Download https://repo1.maven.org/maven2/io/airlift/http-client/0.97/http-client-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/json/0.97/json-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/bootstrap/0.97/bootstrap-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/concurrent/0.97/concurrent-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/node/0.97/node-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/configuration/0.97/configuration-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/discovery/0.97/discovery-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/event/0.97/event-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/floatingdecimal/0.2/floatingdecimal-0.2.jar
Download https://repo1.maven.org/maven2/io/airlift/http-server/0.97/http-server-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/jaxrs/0.97/jaxrs-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/jmx/0.97/jmx-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/jmx-http/0.97/jmx-http-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/log/0.97/log-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/log-manager/0.97/log-manager-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/stats/0.97/stats-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/trace-token/0.97/trace-token-0.97.jar
Download https://repo1.maven.org/maven2/io/airlift/units/0.97/units-0.97.jar
:cleanAssets UP-TO-DATE
:clean
:nodeSetup UP-TO-DATE
:installAssets
npm WARN package.json<EMAIL_ADDRESS>No repository field.
npm WARN package.json<EMAIL_ADDRESS>string_decoder is also the name of a node core module.

<EMAIL_ADDRESS>install /home/airpal/src/main/resources/assets/node_modules/bufferutil
node-gyp rebuild

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/0.10.33"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/home/airpal/src/main/resources/assets/node_modules/bufferutil/.node-gyp"
make: Entering directory /home/airpal/src/main/resources/assets/node_modules/bufferutil/build'   CXX(target) Release/obj.target/bufferutil/src/bufferutil.o   SOLINK_MODULE(target) Release/obj.target/bufferutil.node   SOLINK_MODULE(target) Release/obj.target/bufferutil.node: Finished   COPY Release/bufferutil.node make: Leaving directory/home/airpal/src/main/resources/assets/node_modules/bufferutil/build'
npm WARN prefer global<EMAIL_ADDRESS>should be installed with -g

<EMAIL_ADDRESS>install /home/airpal/src/main/resources/assets/node_modules/contextify
node-gyp rebuild

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/0.10.33"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/home/airpal/src/main/resources/assets/node_modules/contextify/.node-gyp"
make: Entering directory /home/airpal/src/main/resources/assets/node_modules/contextify/build'   CXX(target) Release/obj.target/contextify/src/contextify.o   SOLINK_MODULE(target) Release/obj.target/contextify.node   SOLINK_MODULE(target) Release/obj.target/contextify.node: Finished   COPY Release/contextify.node make: Leaving directory/home/airpal/src/main/resources/assets/node_modules/contextify/build'
npm WARN prefer global<EMAIL_ADDRESS>should be installed with -g

<EMAIL_ADDRESS>install /home/airpal/src/main/resources/assets/node_modules/utf-8-validate
node-gyp rebuild

gyp WARN EACCES user "root" does not have permission to access the dev dir "/root/.node-gyp/0.10.33"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/home/airpal/src/main/resources/assets/node_modules/utf-8-validate/.node-gyp"
make: Entering directory /home/airpal/src/main/resources/assets/node_modules/utf-8-validate/build'   CXX(target) Release/obj.target/validation/src/validation.o   SOLINK_MODULE(target) Release/obj.target/validation.node   SOLINK_MODULE(target) Release/obj.target/validation.node: Finished   COPY Release/validation.node make: Leaving directory/home/airpal/src/main/resources/assets/node_modules/utf-8-validate/build'
:buildAssets
[21:53:08] Using gulpfile /home/airpal/src/main/resources/assets/gulpfile.js
[21:53:08] Starting 'browserify'...
[21:53:08] Bundling app.js...
[21:53:08] Bundling plugin.js...
[21:53:16] Bundled plugin.js in 7.6 s
[21:53:27] Bundled app.js in 18 s
[21:53:27] Finished 'browserify' after 18 s
:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.7
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:13: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol:   class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:66: error: cannot find symbol
implements AsyncHttpClient
^
symbol: class AsyncHttpClient
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:227: error: cannot find symbol
public <T, E extends Exception> HttpResponseFuture executeAsync(Request request, ResponseHandler<T, E> responseHandler)
^
symbol:   class HttpResponseFuture
location: class OldJettyHttpClient
/home/airpal/src/main/java/com/airbnb/airlift/http/client/OldJettyHttpClient.java:381: error: cannot find symbol
implements HttpResponseFuture
^
symbol:   class HttpResponseFuture
location: class OldJettyHttpClient
/home/airpal/src/main/java/com/airbnb/airpal/core/execution/ExecutionClient.java:29: error: package org.jetbrains.annotations does not exist
import org.jetbrains.annotations.NotNull;
^
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:6: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol:   class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryInfoClient.java:11: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol:   class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:19: error: cannot find symbol
private final AsyncHttpClient httpClient;
^
symbol:   class AsyncHttpClient
location: class QueryRunner
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:21: error: cannot find symbol
protected QueryRunner(ClientSession session, JsonCodec queryResultsCodec, AsyncHttpClient httpClient)
^
symbol:   class AsyncHttpClient
location: class QueryRunner
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:42: error: cannot find symbol
private final AsyncHttpClient httpClient;
^
symbol:   class AsyncHttpClient
location: class QueryRunnerFactory
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryRunner.java:44: error: cannot find symbol
public QueryRunnerFactory(ClientSessionFactory sessionFactory, AsyncHttpClient httpClient)
^
symbol:   class AsyncHttpClient
location: class QueryRunnerFactory
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryInfoClient.java:39: error: cannot find symbol
private final AsyncHttpClient httpClient;
^
symbol:   class AsyncHttpClient
location: class QueryInfoClient
/home/airpal/src/main/java/com/airbnb/airpal/presto/QueryInfoClient.java:42: error: cannot find symbol
public QueryInfoClient(AsyncHttpClient httpClient, JsonCodec queryInfoCodec)
^
symbol:   class AsyncHttpClient
location: class QueryInfoClient
/home/airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java:57: error: cannot find symbol
import io.airlift.http.client.AsyncHttpClient;
^
symbol:   class AsyncHttpClient
location: package io.airlift.http.client
/home/airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java:139: error: cannot find symbol
public AsyncHttpClient provideQueryRunnerHttpClient()
^
symbol:   class AsyncHttpClient
location: class AirpalModule
/home/airpal/src/main/java/com/airbnb/airpal/modules/AirpalModule.java:175: error: cannot find symbol
@Named("query-runner-http-client") AsyncHttpClient httpClient)
^
symbol:   class AsyncHttpClient
location: class AirpalModule
16 errors
:compileJava FAILED
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':compileJava'.

Compilation failed; see the compiler error output for details.

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED
Total time: 47.727 secs

I was able to get around a similar error by not specifying the prestoVersion. Not sure if it will work but you may want to try it.
Try this:
sudo ./gradlew clean shadowJar

Yup that works for me but when I go to the next step to specify the presto version its fails.

Are you saying the build did not work? After the build, the next two steps are create mysql database and setup reference.yml. After that you do the migrate database step. Which step is failing?

I did the following
1-./gradlew clean shadowJar -Dairpal.useLocalNode   this was sucessfull
Do I need to run the following one ./gradlew -Dairpal.prestoVersion=0.85 (when running this is failing )clean shadowJar or go to the next step creating the SQL database.

If you have node and npm install locally that should be fine and you can go to next step.

Cool.Then I will try and update you if I am successful.

I followed the steps but when I open the browser I just get blank screen

I'm going to guess that your issue is a misconfiguration in reference.yml. I got it working in my environment tonight and all my issues involved not setting up reference.yml and connectivity with my EMR cluster. My other suggestion is to make sure you can connect with presto-cli tool first. I also had some misconfigurations in my config files for presto.

@kyleburke I tried number of times but I get a blank page when I open the UI.I would appreciate if you can share what you have done with the configurations.
Also when you say "make sure you can connect with presto-cli tool first " Do you mean to test it on the EMR cluster ,If that the case I am able to run connect and run some sample queries from presto cli on EMR.

I had same issue if I specify -Dairpal.prestoVersion=0.138

Getting blank screen after running Airpal .  Please suggest steps to resolve this .

PFA  my reference.yml  file

@surendraGaddam99 try this branch: https://github.com/pubnative/airpal/tree/devel

Hi ,
What things needs to be installed and configured as part of PrestoDB  for running Airpal application.
Thanks
Surendra

@surendraGaddam99  I got the same issues. Did you solve this problem?

Hi,
I am also facing same error.. the BUILD SUCCESSFUL when i ran without presto versio. Not sure how it will work further.

@akalakoti I suggest to use Metabase or Redash instead of airpal.


fix: change the order of header files

when I use build-apisix-base.sh to compile openresty-<IP_ADDRESS>. I found an error like this: https://trac.nginx.org/nginx/ticket/2312#comment:4.
And I found a solution in https://nginx.org/en/docs/dev/development_guide.html#include_files. So I wan to change the order of header files.

It's ok, but this module is not used any more


SetTx Illegal Status Arduino Uno

Hi, I am trying to implement your code on this module which has sx1262 chip. When I upload the code, I'm getting the following error.
Can you please help.
begin
debugPrint=0
SX126x_SPI_SELECT=5
SX126x_RESET=6
SX126x_BUSY=7
SX126x_TXEN=-1
SX126x_RXEN=-1
Reset
syncWord=0x1424
SX126x installed
tcxoVoltage=0.00
useRegulatorLDO=0
SetTx Illegal Status

LoRa modules with SX1262/1268 have several options.

Using TCXO(Temperature-Compensated Crystal Oscillator)
SX1262/1268 can use the TCXO.
If the TCXO is used, the XTB pin is not connected.
However, the 6th pin (DIO3) of the SX1262/1268 can be used to power the TCXO.
Explanation for TXCO and antenna control is here.
Ra-01S / Ra-01SH does not use TCXO.

Power supply modes
SX1262/1268 has two power supply modes.
One is that only LDO used in all modes.
Another is that DC_DC+LDO used for STBY_XOSC,FS, RX and TX modes.
Explanation for LDO and DCDC selection is here.
Ra-01S / Ra-01SH use only LDO in all modes.

RF-Switching
In general, use DIO2 to switch the RF-Switch.
However, some tranceiver use an external gpio to switch the RF-Switch.
Ra-01S / Ra-01SH use the SC70-6 integrated load switch to switch between RFO and RFI.
Ra-01S / Ra-01SH use DIO2 to control this.
DIO2 = 1, CTRL = 0, RFC for SC70-6 goes to RF1 -->  Switch to Tx Mode.
DIO2 = 0, CTRL = 1, RFC for SC70-6 gose to RF2 -->  Switch to Rx Mode.

You need to look at the schematic to set these options properly, but it's very esoteric.

Hello!
I am getting the same error... Can you share your solution??

If this change doesn't work, I don't know how to solve it.
/*
  int16_t ret = lora.begin(RF_FREQUENCY,              //frequency in Hz
                           TX_OUTPUT_POWER);          //tx power in dBm
*/

  int16_t ret = lora.begin(RF_FREQUENCY,              //frequency in Hz
                           TX_OUTPUT_POWER,           //tx power in dBm
                           3.3,                       //use TCXO
                           true);                     //use TCXO

It still gives following error:
begin
debugPrint=0
SX126x_SPI_SELECT=5
SX126x_RESET=6
SX126x_BUSY=7
SX126x_TXEN=-1
SX126x_RXEN=-1
Reset
syncWord=0x1424
SX126x installed
tcxoVoltage=3.30
useRegulatorLDO=1
SPI Transaction error:10
SPI Transaction error:10
SetRx Illegal Status
Btw I have not connected any of the TXEN or RXEN pin of LoRa. The DIO pins are also disconnected.

Contacting the LoRa module vendor may provide a solution.

Still not working... I instead used the RadioLib library which is working fine...

Still not working...


ssize_t is undefined

In sys.h file, the macro for Windows system is UDS_SYS_WINDOWS
But in sys_win32.h file, the macro used is UDS_SYS_WIN32
This lead to the error "ssize_t is undefined"

Hi VoLinhTruc, feel free to try out the windows branch here: https://github.com/driftregion/iso14229/pull/35


Fix min/max map functions

Since these work off of per-cpu maps, it means that when a key is added the value for the key across all cpus initializes to 0. Therefore if we loop over all cpus to determine if a value is greater or less than another we need to discount all 0 values that weren't set explicitly by the user/prog (e.g. min(0)).
To do this we need to keep a separate key (similar to avg) that acts like a "value was set" boolean so we know if we should consider it in a less-than or greater-than operation.
This needs to be done in both userspace (for map printing) and in kernel space for min/max implicit casting.
Issue: https://github.com/bpftrace/bpftrace/issues/3286

Checklist

[ ] Language changes are updated in man/adoc/bpftrace.adoc
[ ] User-visible and non-trivial changes updated in CHANGELOG.md
[ ] The new behaviour is covered by tests

This is ugly but correct. The alternative is to have min/max use hash maps and an atomic min/max write after lookup - I don't know what would be faster but I'll try to run some tests to find out.

I don't understand the exact problem here as I can't trigger any wrong results except by using tuples. Left a comment on the issue (#3286).

Responded in the issue with a more detailed repro.
Here is the alternative solution where we just convert min/max to use a hash map, which I'm leaning towards because it's just so much simpler AND preserves the ability to loop over min/max maps. https://github.com/bpftrace/bpftrace/pull/3290

Abandoning in favor of https://github.com/bpftrace/bpftrace/pull/3298, which stores a struct as the min/max map values instead of using two separate keys.


Port a the SoR History fix to Stable Cadence

Issue To Be Solved
Port https://github.com/onflow/flow-core-contracts/pull/416 to https://github.com/onflow/flow-core-contracts/tree/stable-cadence

Completed in 1b20b4d


Actions

Hey @Karanuchiha , I'm going to close this, so you can start fresh on a new branch with the rename and restructure.


ZERO Hydra Failures 20.09

Jobsets:

https://hydra.nixos.org/jobset/nixos/release-20.09
https://hydra.nixos.org/jobset/nixos/release-20.09-aarch64
https://hydra.nixos.org/jobset/nixos/release-20.09-small
https://hydra.nixos.org/jobset/nixpkgs/nixpkgs-20.09-darwin
https://hydra.nixos.org/jobset/nixpkgs/staging-20.09

Mission
Every time we branch off a release we stabilize the release branch.
Our goal here is to get as little as possible jobs failing on the release-20.09 jobsets.
I'd like to heighten, while it's great to focus on zero as our goal, it's essentially to
have all deliverables that worked in the previous release work here also.
How many failing jobs are there?
At the opening of this issue we have the main x86_64-linux jobset at 1153 failing jobs, x86_64-darwin at >7130, and aarch64-linux at 7573+.
Previous releases
The first evaluation of 19.09 had 1654 failing jobs.
The first evaluation of 20.03 had 1204 failing jobs,
The first evaluation of 20.09 had 1153 failing jobs,
So we're actually getting better at maintaining a more stable "unstable" channel.
However, our darwin story isn't as good (we need more darwin reviewers/contributors)
The first evaluation of 20.03 had 1384 failing jobs,
The first evaluation of 20.09 had >7130 failing jobs,
How to help (textual)

Select an evaluation of the release-20.09 jobset by #id

Find a failed job ❌

Troubleshoot why it's failing and fix it

Create a Pull Request with the fix targeting master, wait for it to be merged.
Generally the job fails on master also, you can verify that on Hydra - example URL: https://hydra.nixos.org/job/nixpkgs/trunk/bash.x86_64-linux.
That means most PR's should be target the master branch, however, if your PR causes around 500+ rebuilds, it's preferred to target staging to avoid compute and storage churn.

Always reference this issue in the body of your PR:
ZHF: #97479

After the master PR was merged, select the commit(s) from the master branch and cherry-pick the commit(s) into a backport PR. [See also, (https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md#backporting-changes). An example can be found here: Master PR and 20.09 PR

Please ping @NixOS/nixos-release-managers on the PR.
How to help video tutorial
@jonringer has made a video on YouTube to guide anyone through how fixing something for ZHF will look like: https://www.youtube.com/watch?v=4Zb3GpIc6vk&
New to nixpkgs?
@jonringer created some videos to help get people started with nixpkgs:

Packaging a basic C application
Python nix packaging
Adding a package to nixpkgs
Backporting changes (ZHF)

Also be sure to check out other resources at: https://github.com/nix-community/awesome-nix
Packages that don't get fixed
The remaining packages will be marked as broken before the release (on the failing platforms).
You can do this like:
meta = {
  # ref to issue/explanation
  # `true` is for everything
  broken = stdenv.isDarwin
};

These are the utility flags used to test the type of platform.
Closing
This is a great way to help NixOS, and it was some of my earliest contributions.
Let's go ✌
✨ @worldofpeace and @jonringer
cc @NixOS/nixpkgs-committers @NixOS/nixpkgs-maintainers
Related Issues

Timeline: #95535
Feature Freeze Items: #95475
Release Notes: #95765

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nixos-20-09-zero-hydra-failures/8928/1

It can also be useful to refer to https://hydra.nixos.org/eval/1611944, a master evaluation taken around the time of the branchoff, which will have a much longer history you can use to dig back to a package's last successful build and get an indication of the changes which might have caused it to break.

Also, people can use @samueldr 's nix-review-tools to create a report which will show which packages are causing the most failures.
I usually do something like:
mkdir cache
cd cache
../eval-report $evaluation_id > index.html
xdg-open index.html

The nearest NixOS:trunk-combined Hydra evaluation is 1611942 at commit 6152513, 6 commits after the last shared commit 53ce0bf.

However, our darwin story isn't as good [...]

It's blown up by ghc timing out.  That has been occasionally happening and it should go away after some restart(s).

@vcunat should we pass "big-parallel" for the ghc build so that it has more cores?

Since last time this was useful to many, here's one way to build all packages which have you as maintainer:

apply #97514
run nix-build maintainers/scripts/build.nix --argstr maintainer fgaz (adjust the maintainer name to yourself)

We cannot ping @NixOS/nixos-release-managers as required in the issue if we’re not part of the organization, is it possible to fix that?

I know there's a more limited number of people who use nix on macOS, so I'd like to focus my efforts there, is there a way of getting a list of jobs that are succeeding on nixpkgs/staging-20.09 and failing on nixpkgs/nixpkgs-20.09-darwin?

@uri-canva that is possible by opening an evaluation of nixpkgs/nixpkgs-20.09-darwin and the you have to hit the Compare to button and select nixpkgs/staging-20.09. All "Newly failing" builds are fine one staging-20.09, but fail on nixpkgs-20.09-darwin.
However it should be sufficient to filter for jobs matching x86_64-darwin in Search jobs by name, right?

Yes, but that would also show derivations that are failing for the same reason on linux and darwin, and would be fixed by fixing the linux derivations anyway.

Is there any way to get a list of my (currently broken) derivations like in some previous ZHF sprints?

@makefu you could try to build them with the command i posted in this thread

As an aside, I wish we could opt-in to failed build notification emails from hydra. I didn't know some of my packages were broken :-/

@makefu you could try to build them with the command i posted in this thread

I actually tried that but the build it looks like the process fails pre-build. I am now in the progress of trying to fix all evaluation errors
~/nixpkgs git reset --hard upstream/master
HEAD is now at 5d131d33268 Merge pull request #97540 from danieldk/fix-clpeak
~/nixpkgs nix-build maintainers/scripts/build.nix --argstr maintainer makefu --show-trace --keep-going
error: while evaluating the attribute 'drvPath' at /home/makefu/nixpkgs/lib/customisation.nix:163:7:
while evaluating the attribute 'buildInputs' of the derivation 'python2.7-aresponses-2.0.0' at /home/makefu/nixpkgs/pkgs/development/interpreters/python/mk-python-derivation.nix:108:5:
while evaluating 'getOutput' at /home/makefu/nixpkgs/lib/attrsets.nix:464:23, called from undefined position:
while evaluating anonymous function at /home/makefu/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:143:17, called from undefined position:
while evaluating 'callPackageWith' at /home/makefu/nixpkgs/lib/customisation.nix:117:35, called from /home/makefu/nixpkgs/pkgs/top-level/python-packages.nix:5446:20:
while evaluating 'makeOverridable' at /home/makefu/nixpkgs/lib/customisation.nix:67:24, called from /home/makefu/nixpkgs/lib/customisation.nix:121:8:
while evaluating anonymous function at /home/makefu/nixpkgs/pkgs/development/python-modules/pytest-asyncio/default.nix:1:1, called from /home/makefu/nixpkgs/lib/customisation.nix:69:16:
while evaluating 'makeOverridablePythonPackage' at /home/makefu/nixpkgs/pkgs/top-level/python-packages.nix:36:37, called from /home/makefu/nixpkgs/pkgs/development/python-modules/pytest-asyncio/default.nix:2:1:
while evaluating 'makeOverridable' at /home/makefu/nixpkgs/lib/customisation.nix:67:24, called from /home/makefu/nixpkgs/pkgs/top-level/python-packages.nix:38:12:
while evaluating anonymous function at /home/makefu/nixpkgs/pkgs/development/interpreters/python/mk-python-derivation.nix:31:1, called from /home/makefu/nixpkgs/lib/customisation.nix:69:16:
pytest-asyncio-0.14.0 not supported for interpreter python2.7

I'll be back once i finished that and maybe encountered some actual build errors.

Since last time this was useful to many, here's one way to build all packages which have you as maintainer:
* ~apply #97514~ Pull from master so you have the latest build.nix mentioned below

* run `nix-build maintainers/scripts/build.nix --argstr maintainer fgaz` (adjust the maintainer name to yourself)

@fgaz won't this build on master? Don't we want to test the nixos-20.09 branch?

@bbigras well we're trying to fix both! But yes, I see what you mean, and I guess that pr needs a backport. I'll open another pr.

@makefu I got a fix for you  #97571
I ran into the same issue with python packages where unsupported interpreters throw an error

I've tried fixing springLobby but it fails to find libcurl, did somethng change about that around 15th August?

Thank you for clearly describing the contrib process for ZHF. I drive-by fixed a package I happened to use, but wouldn't have bothered contributing to ZHF if it wasn't so easy.

I found a few cases in perlPackages builds on Darwin where there's an issue with ld which is fixed by adding "export LD=$CC", which several packages appear to have done already. Some predicate on i686 or Darwin, and some do it unconditionally.
Is it worth opening an issue and trying to find a more general fix for all perlPackages, or should I just do one-off PRs for each one I find like this?
Or maybe do them both so there's at least a fix for 20.09?

@volth , do you know the best way going forward for @treed

nix-build maintainers/scripts/build.nix --argstr maintainer fgaz

Is there a way to get past evaluation errors if there are some? (I get an error: bcrypt-3.2.0 not supported for interpreter python2.7)

What is to be done if a package is broken in release-20.09, but works in master? My concrete case is kmime; it was broken by commit ce4eb0b79b3b8e830d40345fb6457fac6ca9a9ec and still is in release-20.09, but is fine in master.

What is to be done if a package is broken in release-20.09, but works in master? My concrete case is kmime; it was broken by commit ce4eb0b and still is in release-20.09, but is fine in master.

A backport I would guess.

I'd use nix show-derivation -f. kmime > kmime.$(git describe HEAD).json when the master and the release branch are checked out, and diff the outputs.

Is there a way to get past evaluation errors if there are some?

@knedlsepp either #97571 or #97647

What is to be done if a package is broken in release-20.09, but works in master? My concrete case is kmime; it was broken by commit ce4eb0b and still is in release-20.09, but is fine in master.

@ttuegel see comments in #97242 for why kmime is failing

@thoughtpolice in january you said (in https://github.com/NixOS/nixpkgs/pull/77985#pullrequestreview-344972610):

I would also be fine with dropping FoundationDB 5.x builds, too, since they're pretty heavy to build and I imagine most people are running 6.x at this point.

Do you think now might be the time to do that?

pythonPackages.gipc looks like it'll be stuck waiting on https://github.com/jgehrcke/gipc/issues/103

I was thinking about going through and disabling all the failing python27Packages that have explicitly dropped python 2 upstream, but I see this https://github.com/NixOS/nixpkgs/pull/92099. Is there any point in bothering? @jonringer

I was thinking about going through and disabling all the failing python27Packages that have explicitly dropped python 2 upstream, but I see this #92099. Is there any point in bothering? @jonringer

I've been somewhat doing that while reviewing other packages https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+author%3Ajonringer+disable+is%3Aclosed+python
The discussion you linked to is a bit more involved.
Not recursing into the attr set is the "easiest" solution, but will not perfect for users that still need some python2 packages

I am a bit late and don't know if this is the right place as I am new to ZHF but I had a look at some darwin packages with the help of nixpkgs-review-tools:

Name
Count
Builds Locally
Builds on Hydra
Action Needed

python38-curio-1.2
112
:heavy_check_mark:
:x: Socket Address already in use
Builds fine locally, don't know how to fix this for hydra? Just retry it?

qtbase-5.14.2
95
:x:
:x: Cannot find feature sdk
@eqyiel https://github.com/eqyiel/nixpkgs/commit/50c2b5fd030459ff9508f65e9ffdebad0de36a63 has a WIP that removes the usr/bin/xcodebuild impurities. Unfortunately it needs at least the 10.13 sdk. See https://github.com/NixOS/nixpkgs/issues/95199 for more details. @matthewbauer Would now be a good time to upgrade the sdk? Probably that is too much work but is it possible to have multiple apple sdk versions?

qtbase-5.15.0
47
:x:
use of undeclared identifier
not looked at maybe the 5.14.2 actions fix this too

python3.7-notebook-6.1.3
40
:x:
:x: Insufficient access privileges for operation delete
I fixed this https://github.com/NixOS/nixpkgs/pull/98486 and backport is still waiting for review here https://github.com/NixOS/nixpkgs/pull/98621

python3.8-notebook-6.1.3
38
:x:
:x: same as python37
fix of python37 should also fix this

python3.8-fsspec-0.7.4
27
:heavy_check_mark:
:x: def test_modified fails
Very strange. Have a look at the failing test in the log. It creates a file, modifies it and checks if the modified date is after the creation date. Don't know why that fails on hydra. Any ideas?

python3.8-fs-2.4.11
18
:x:
:x: resource tracker cannot free resource
Seems to be a multiprocessing issue with python 3.8.5 on macosx as it works with python37. I opened an issue here https://github.com/PyFilesystem/pyfilesystem2/issues/430 and disabled the tests in this pr https://github.com/NixOS/nixpkgs/pull/98619 which is open for review too.

@tricktron re curio, I think it would be perfectly acceptable to disable these two tests on darwin given the impact. It's clearly not an issue with the library.

I also keep finding someone has got ahead of me discovering upstream issues (frequently @jonringer) so I think it might be useful to link to upstream blocker issues here to save people digging down to the same issue.

pythonPackages.localzone is held up on https://github.com/ags-slc/localzone/issues/1

(there's also a chance the author seeing the issue referenced might get spurred into action...)

Samba build is fixed on darwin in 4.13+ by
https://gitlab.com/samba-team/devel/samba/-/commit/847208cd8ac68c4c7d1dae63767820db1c69292b
which has also been backported in 4.12.6. I also backported the patch to apply cleanly to the current 4.12.5 to avoid rebuilds, but would it be preferable to just update to 4.12.6? (New to ZHF so I don't know what impact that would have)

@tricktron re curio, I think it would be perfectly acceptable to disable these two tests on darwin given the impact. It's clearly not an issue with the library.

@risicle Ok pr opened here https://github.com/NixOS/nixpkgs/pull/98875

Also, notebook backport has been approved.

Thanks

@r-burns I'd personally go for the bump.

git-annex-adapter is stuck waiting on https://github.com/alpernebbi/git-annex-adapter/issues/13 and https://github.com/alpernebbi/git-annex-adapter/issues/14
(after getting past the immediate trivial build-stopper by passing cacert to checkInputs)

/me coughs in the direction of #98688

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/is-there-a-feasible-way-to-debug-test-hydra-darwin-failures-quickly/9201/1

PRs still needing review in this thread:
#100049
#99673
#99655
#99642
#99586
#99500
#99383
#99252
#99076
#99052
#99040
#98580
#98381

yea, sorry, mixture of burnout and ck3 has lead me to neglecting this

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nixos-weekly-08-nixos-weekly/9431/1

@jonringer went through all of open ones and merged most of them :)

Many others also helped, I'll do a proper thanks in the release post :)

The x86_64 build of itk-5.1.1 is broken on Hydra with an "Illegal instruction" error, but building locally succeeds (and Hydra builds master), so I'm not exactly sure how to bisect/troubleshoot this.

The x86_64 build of itk-5.1.1 is broken on Hydra with an "Illegal instruction" error, but building locally succeeds (and Hydra builds master), so I'm not exactly sure how to bisect/troubleshoot this.

"Illegal instruction" means a program uses an instruction that is unsupported by a CPU, Hydra builders have different CPUs so certain instructions will be supported only by some machines.
This happens because ITK is built with -march=corei7 (https://github.com/InsightSoftwareConsortium/ITK/blob/4b48c9025f66d179d7b134999e2398f5924093b4/CMake/ITKSetStandardCompilerFlags.cmake#L245). -march=corei7 means that the program is compiled for CPUs supporting MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1 and SSE4.2 only.
I believe we may want to remove -march=corei7 (because it's not correct for NixOS) and -mtune=native (because reproducibility).

What's the plan for theano? @twhitehead had several suggestions in https://github.com/NixOS/nixpkgs/pull/99516#issuecomment-703289554 but which do we want to take forward for release-20.09?

Hmm... I quite need #99587 merged to be able to backport it to 20.09, but its dependencies are now broken on master.

Hmm... I quite need #99587 merged to be able to backport it to 20.09, but its dependencies are now broken on master.

Assuming the dependencies aren't broken on 20.09, would it be permissible in this extraordinary case to either

make the backport already now, although #99587 isn't merged yet

or to

implement the fix directly on 20.09 and later forward-port it to master

?

I've essentially made the backport already because it requires some subtle tweaks on release-20.09. It's just it would feel pretty wrong for something that includes a new package to make it to a stable branch before master.

  We currently have a fixup in staging-next, so if it turns out well on Hydra, I planned to pick it to 20.09 as well.  (other platforms won't even rebuild)

I expect that KDE/Qt failures might be some remaining cases of combining different Qt5 versions, e.g. /cc https://github.com/NixOS/nixpkgs/pull/101411
metrics is my problem, please disregard it for now (I trust the outage doesn't cause problems to anyone).

Regarding the Fritzing build error, it seems I forgot to backport https://github.com/NixOS/nixpkgs/pull/98381

It didn't build, now it starts up OK.  Picked as f5ad6d9f77.

 Thus this should be closed.
Fixing broken packages will always be possible throughout the lifetime of the 20.09 release. However, you may need to remove the broken = true; attr on the package.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/helping-with-hydra/9788/4

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/20-09-release-retrospective/9851/19


How to downgrade the yocto version with meta-ros

Here I have to use yocto 3.1.0 to build meta-ros1-melodic for some usage.
What files should I need to edit to fetch the correspond yocto version with your SOP in instruction ?

What you mean? You cannot use newer dunfell branch revision than the initial 3.1.0? Why? 3.1.0 is missing many important changes and fixes.

Because my hardware i500’s newest yocto version is yocto3.1.0. I have no choice but to use this version to build meta-ros.

Sorry, I don't know what i500 is or why it would be incompatible with 3.1.10 (latest revision of dunfell release as in https://lists.yoctoproject.org/g/poky/message/12511).

This doesn't appear to be a bug and is over 2 years old.


Unable to Install Starter

It appears not possible to install this starter in its current form. This has been attempted on multiple devices, with multiple people. When 'gatsby develop' is run, this error is shown:
Steps to Reproduce:
Given Node.js and Git are installed...

npm install -g gatsby-cli
gatsby new project-name https://github.com/akzhy/gatsby-starter-elemental
cd project-name
gatsby develop

Hey could you try it by
git clone https://github.com/akzhy/gatsby-starter-elemental
cd gatsby-starter-elemental
yarn
yarn develop

Seemingly unrelated to this, when I locally run the starter, it doesn't seem as thought the front page of the starter is affected by any of my changes to config.ts. While editing other files result in changes in the corresponding page, editing config.ts doesn't lead to the same change.

Did you try restarting the dev process?

Yes, I've tried this and it still does not work. I've also replicated the issue on different devices.

Are you able to edit config.ts and see a change in the site?

Did you try restarting the dev process?

I have now. Thank you. I previously interpreted what you said to be reinstalling the starter, I now realise that you meant rerunning the server. I stopped running it in the browser, made a change, and reran it, and it worked perfectly, thanks.


⚠ Turnos 24x7 has degraded performance

In 8d4c206, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 924 ms

Resolved: Turnos 24x7 performance has improved in 6ab9cec.


Why do property.Readable and property.Writable default to true with memberserialization == MemberSerialization.Fields?

Issue
I'm curious what value it adds?  Clearly it's there for a specific purpose.  It's confusing though because if a property has only a 'get' it will get set to 'writable'.
Code
//line 1195 in DefaultContractResolver
if (memberSerialization != MemberSerialization.Fields)
{
property.Readable = ReflectionUtils.CanReadMemberValue(member, allowNonPublicAccess);
property.Writable = ReflectionUtils.CanSetMemberValue(member, allowNonPublicAccess, property.HasMemberAttribute);
}
else
{
// write to readonly fields
property.Readable = true;
property.Writable = true;
}

I don't understand what you're asking. It does that because that's how fields serialization works.

The question is simply this: why would a readonly property (e.g. public string Name{get;}) get marked as 'Writable=true' when using MemberSerialization.Fields?  The backing field may be writable but the property itself is readonly.  Just curious really.  Thanks for your time.

@BlackjacketMack I was also surprised when I originally learned that the backing fields generated for read-only properties are not given the readonly keyword. I have no idea why.

@sungam3r Well, the readonly modifier seems perfectly sensible for the backing fields of get-only auto-properties. If nothing else, having it makes more sense than not having it when inspecting the field through reflection. It could be that it "just wasn't done", as I believe you are implying. Still, it doesn't strike me so much as "having to do something", but rather as a choice between "should we generate the backing field with or without the readonly modifier"?
Why, then, did they choose to omit readonly? My guess is that the modifier was omitted as a performance optimization for value types. For value types, something like this._creationDateTime.AddDays(1) involves an extra copy of the struct if the _creationDateTime field is readonly. Why the extra copy? Because we cannot easily guarantee that AddDays() avoids mutating this, but a readonly field must not be mutated. (Clearly this does not apply to readonly structs, now that we have those. I'm guessing that the backing field generation was simply kept consistent between auto-properties for mutable structs, readonly structs, and classes, i.e. the backing field is simply never made readonly.)


Multi node tests in CI

We should add some tests that use a cluster of multiple nodes.
For example:

Make a cluster of 3 scylla nodes
Make one node unreachable from the driver (drop all packets from driver to this node)
run cargo test

This would prevent bugs like: #306
It looks like java-driver uses ccm to run tests in their CI, we could do something similar.
https://github.com/scylladb/java-driver/blob/scylla-3.x/.github/workflows/tests.yml#L26

is ccm required? I guess a simple docker compose file, mention in #306, would also work. When a node is added to the cluster while this driver is running in an application, does this driver adds the new node in memory? Maybe that can be tested as well while fixing this issue

In https://github.com/scylladb/scylla-rust-driver/pull/489, we modified the CI to run tests on a 3-node cluster. We can additionally consider tests which will modify the cluster while they are running to tests how the driver behaves when the topology changes.

Closing in favor of https://github.com/scylladb/scylla-rust-driver/issues/1097


Not work with react-select-plus correct

I have both packages in my project. So when i click react-datepicket it is open. But when click to react-select-plus. It is not open and not gain focus.  Same happen if your open this codesandbox. Open first calender then click editor. Your cursor not moved from calender component.
Expected behavior
When click other input and component it should close itself
Actual behavior
When click other input and component it is not closing.
Steps to reprodcue
Click calender, then click react-select-plus select component.

#1170 duplicate it sorry


notify the state in checkForPartialScroll

I d like to know the state is Scrolling inside the animation of  checkForPartialScroll
viewDidLayoutSubviews is not following the changes from the followers.
hope this PL is okay to be merged

Sure, sounds reasonable, thanks!


photo & file system (mobile)

Allows user to decide to take a new photo (mobile) or use the file system of the OS.

minor - Foto dreh icon is blank in Firefox

added #18


vue-tel-input@next is missing dropdownOptions.showSearchBox props

On vue-tel-input@next missing dropdownOptions.showSearchBox. And there is no way to search for the desired country

HI folks, after update to vers. 8.1.0 (npm install vue-tel-input@latest), a 'showSearchBox' is still missing. In node modules -> vue-tel-input have only style.css (not vue-tel-input.css).
Any ideas ?

thanks @Diarten. yep, it should be fixed in the latest (8.1.1).
sorry guys I have been migrating the build tool to vite, which created some hiccups. the documentation page is also updated using vitepress: https://vue-tel-input.iamstevendao.com/. I'll close it but feel free to reopen if it's still not working.

@chinhdhse02750 do you use the vue-tel-input@next version?
I just use the latest version 8.1.1 (not  vue-tel-input@next ) with vue3 and it works well.
import VueTelInput from 'vue-tel-input';
import 'vue-tel-input/dist/vue-tel-input.css';
...
app.use(VueTelInput);
https://vue-tel-input.iamstevendao.com/guide/installation.html


The receiver resets the session on TesterPresent

Found a bug in this code. The session was reset rather from Supress-Pos-Response rather then Tester-Present.

Thanks for the PR.
The session features has rarely been used I think.
I don't really understand the original bug, though.
The only change I see is that the session timer is now reset on every request instead of only on requests actually defined within the "Raw" section.


plot-all.sh

The following error occurs when I run plot-all.sh. How do I resolve it?

After the above problem is solved, all the drawings drawn are blank pdf files.

we should upgrade gnuplot to a more recent version.


feature: add workflow for building docker image

Where is the docker image?

Where is the docker image?

We don’t have time to learn how to create a docker image, so we might as well roll back the code.


Allow Checkboxes in DataTables to inherit colors from CheckboxTheme

This PR fixes https://github.com/flutter/flutter/issues/96004
This PR resolves a stale TODO left in the DataTable code (see the issue for details), by removing the hardcoded values from DataTable controlling the checkboxes colors, which instead allows them to inherit those values for the checkColor and fillColor from the CheckboxTheme rather than ColorScheme.primary and ColorScheme.onPrimary.
Before and after:

Pre-launch Checklist

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I read the [Tree Hygiene] wiki page, which explains my responsibilities.
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I signed the [CLA].
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is [test-exempt].
[x] All existing and new tests are passing.

Although, this would need another reviewer's approval to land, as per our new commit policy.

Great, sounds good, thanks for the review! Any suggestions for how I should go about trying to get another pair of eyes on it? If the best practice is to just wait until another reviewer finds it among the list of PRs, that's totally fine with me, just not exactly sure what the etiquette/expectation is for how that usually works 👍🏻

Although, this would need another reviewer's approval to land, as per our new commit policy.

Great, sounds good, thanks for the review! Any suggestions for how I should go about trying to get another pair of eyes on it? If the best practice is to just wait until another reviewer finds it among the list of PRs, that's totally fine with me, just not exactly sure what the etiquette/expectation is for how that usually works 👍🏻

PRs should get reviewed within about a week when the relevant team's PR review meeting happens (might be a bit longer with the holiday in the US right now).


Converted dvtm-cmd.1 and dvtm.1 to mandoc format

Maybe you do not want to pull dvtm-cmd.1 directly.  You also can pick only files you want or ask me to remove some.
I checked both with mandoc -T lint to check errors and none remain.  Are these looking ok to you?

Thanks! I merged the dvtm(1) man page, but left out dvtm-cmd(1) for now because it is currently not distributed with dvtm.
If you could review the existing man page for missing key bindings and/or add man pages for the new dvtm-editor(1) and dvtm-pager(1) that would be appreciated.

Yes, better let the dvtm-cmd.1 for when the script will be in tree.
I'm now having a look at dvtm-editor(1) and dvtm-pager(1).  Thank you for implementing them.


Multiple dialogs sometime open when migrating a pipeline

Describe the issue
Sometime the migrate pipeline dialog is opened multiple times. This is a UX issue only since the functionality is still only run once, the extra dialogs only confuse the user and don't affect the pipeline migration.
To Reproduce
Steps to reproduce the behavior:

Open a pipeline created with an older pipeline version
The migrate pipeline dialog will open, click a button
A migrate pipeline dialog will still be open. If you selected migrate button the migration will have happened behind the dialog. If you selected cancel then the pipeline will already be closed.
If you click migrate again nothing happens, but if you click cancel it will try to close the pipeline and you would be prompted to save.

Expected behavior
The migrate dialog should only show once and no second dialog should be seen once a choice is made

Afaik the onError function that handles the migration prompt is being called multiple times in pipeline-editor. This is most likely because it is called within this useEffect https://github.com/elyra-ai/pipeline-editor/blob/f1360ee09f424cf13ca1609ff67663468e01a5ee/packages/pipeline-editor/src/PipelineEditor/index.tsx#L211 which is most likely being triggered more than once.

Would this be related to #863 ?

You need to wrap onError in useCallback to memoize it, otherwise onError will be a new variable every render which can cause unnecessary useEffect re-runs

You need to wrap onError in useCallback to memoize it, otherwise onError will be a new variable every render which can cause unnecessary useEffect re-runs

Ah that makes sense, I'll fix this once I'm done writing the tests for https://github.com/elyra-ai/pipeline-editor/pull/147

After some research and discussion with @bourdakos1 this is a symptom of a larger issue that will require a refactor of a lot of our hooks. So we will address this in 3.1 instead. Specifically the use of SWR is causing a lot of hooks to be run over and over when it isn't necessary, one symptom of which is that the update of runtimeImages from unknown to it's actual value via SWR updates pipeline which reruns open, which errors again and opens a migrate dialog even though one is already open.

So we will address this in 3.1 instead.

This seems significant.  I'm assuming you meant 3.0.1 based on the milestone update.

Specifically the use of SWR is causing a lot of hooks to be run over and over when it isn't necessary

What is SWR and was its use introduced only in 3.0?
Besides the update of runtime images and since those are general metadata instances, does this issue apply to all metadata editor operations?  I.e., should we enumerate all affected areas?

Besides the update of runtime images and since those are general metadata instances, does this issue apply to all metadata editor operations? I.e., should we enumerate all affected areas?

@kevin-bates I believe this is only affecting the pipeline editor extension at the moment.
To clear things up a bit, SWR itself isn't exactly the issue, it's more that our PipelineEditor Widget isn't equipped to handle dynamic data yet. We need to refactor some of our current logic/assumptions so that it is more robust to these types of data changes

Thanks @bourdakos1 - that helps a lot.  And SWR is a strategy for data fetching apparently (and sounds interesting).