Pangeo / earth science workflows

There is a large portion of dask.array users who use Dask in geospatial / climatology / earth science / etc. use cases. We should make sure these use cases are represented here in our benchmarks.
@TomNicholas @dcherian we're trying to establish a set of benchmarks for common workloads faced by Dask users. Ideally these are more realistic and less toy examples. These get used day-to-day by engineers as they run AB tests on various changes that they want to make. Workloads in this suite will naturally be optimized over time. What are 1-3 workflows you'd most like to see run regularly as part of Dask's benchmark suite?
xref https://github.com/coiled/coiled-runtime/issues/725

How would you like the code to be written/contributed?

Presumably some kind of small/big version of the problem, controlled by a small number of parameters?
Can xarray be a dependency?

I think some combo of groupby/groupby-anomaly (exercise shuffling, reductions); then rolling/rolling anomaly (to exercise map_overlap) is probably what we want.
@aulemahal / @huard do you have a prototype xclim workflow that would be good to test?

How would you like the code to be written/contributed?

Ideally there's some Jupyter notebook somewhere that someone could point us to. Maybe you already have a notebook or two in mind?
Ultimately we want to add the workflow in this repo as a pytest-style test (here's an example of an existing embarrassingly parallel workflow). These tests are run regularly and used in AB tests as we're developing dask and distributed. Eventually these workflows will probably graduate into an example repo with notebooks that people could checkout, play with, and adapt to their own use cases.

Idealized dataset?
Presumably some kind of small/big version of the problem, controlled by a small number of parameters?

We're able to use large-scale clusters on the cloud here, so large, realistic datasets are welcome. For example, that embarrassingly parallel workflow processes ~3 TB of data on AWS.
In terms of complexity we'd love something in between 20-200 lines of code. We're trying to strike a balance between these workflows being representative of what large Dask user groups actual do, while also making them approachable (i.e. we don't want people's eyes to glaze over when they see 1000 lines of code).

Can xarray be a dependency?

Yeah, definitely. Workflows using dask or xarray (backed by dask) are totally welcome.

Hi @jrbourbeau - this sounds like a good idea.

We're able to use large-scale clusters on the cloud here, so large, realistic datasets are welcome.

What about requester-pays data?

Can xarray be a dependency?

Yeah, definitely. Workflows using dask or xarray (backed by dask) are totally welcome.

How about xGCM or xhistogram? They both ultimately just wrap xarray / dask, but produce task graphs with structures that conventional xarray usage doesn't. (If not that's okay, it would just make a neat example trickier.)
@jbusecke might also have some suggestions

Hi @jrbourbeau - this sounds like a good idea.

We're able to use large-scale clusters on the cloud here, so large, realistic datasets are welcome.

What about requester-pays data?

Can xarray be a dependency?

Yeah, definitely. Workflows using dask or xarray (backed by dask) are totally welcome.

How about xGCM or xhistogram? They both ultimately just wrap xarray / dask, but produce task graphs with structures that conventional xarray usage doesn't. (If not that's okay, it would just make a neat example trickier.)
@jbusecke might also have some suggestions

What about requester-pays data?

Yeah, requester-pays is fine. We can spin up a cluster and pay for data access anywhere on AWS (and GCP soon). For example, that embarassingly parallel example I referenced earlier is processing ~3TB with requester_pays=True.

How about xGCM or xhistogram?

Good question. I'm inclined to stick with just dask and xarray dependencies at the moment as they're more broadly used

I support @TomNicholas suggestions from above.
I think including some sort of vector calculus operator like div/curl would be incredibly valuable in the long run. This sort of operation usually works fine with toy examples but can turn out to be really tricky when running under real-world conditions due to:

combining several data_arrays of different size (e.g. full 4D array of velocities and a 2D array of cell area)
not reducing output dimensionality via aggregation (e.g. output data-arrays are the same size as input)
array padding, which increases the amount of comms between workers

Looking through some of the existing pangeo example, I'm wondering if something like https://gallery.pangeo.io/repos/pangeo-gallery/physical-oceanography/01_sea-surface-height.html or https://gallery.pangeo.io/repos/pangeo-gallery/cmip6/global_mean_surface_temp.html would be considered representative. @dcherian @TomNicholas @jbusecke do those notebooks look like common operations the pangeo community cares about?

Those seem too "easy" at a quick glance.
Here's one https://github.com/pangeo-data/distributed-array-examples/issues/2 . Gabe and I discussed this a bit in https://github.com/pangeo-data/distributed-array-examples/issues/2
This point

I think there is a small but important increase in complexity here because we do ds.u.mean(), ds.v.mean(), (ds.u*ds.v).mean() all together so each chunk of ds.u and ds.v is used for two different outputs.

is definitely a common "feature" of Pangeo workloads.

This one was fixed by worker saturation for a single year of data. Not sure how well it does for all 60 years of data.
The example there uses 20 years, it's taking  quite a while for the computation to start.

Finally a  pangeo classic, anomalies with respect to a group mean: https://github.com/pangeo-data/distributed-array-examples/issues/4 .
I don't have time to try it now, but excited to see what you coilies figure out :P

Thanks folks.  I'm curious, the benchmarks currently run on AWS.  Are there AWS variants of the datasets linked to above by any chance?  We can also run on GCP, it's just more flags/keywords to set and billing accounts to keep track of.

If I take https://github.com/coiled/benchmarks/issues/770#issuecomment-1506254519 from @dcherian above and swap out 20 years for 1 year it seems not terrible.  It's about a 20s startup time (which I suspect may have as much to do with da.random.random as anything else).
It's pretty much entirely network bound:

import pandas as pd
import xarray as xr

import dask.array
from dask.distributed import wait
import flox.xarray

dims = ("time", "level", "lat", "lon")
# nyears is number of years, adjust to make bigger,
# full dataset is 60-ish years.
nyears = 1
shape = (nyears * 365 * 24, 37, 721, 1440)
chunks = (24, 1, -1, -1)

ds = xr.Dataset(
    {
        "U": (dims, dask.array.random.random(shape, chunks=chunks)),
        "V": (dims, dask.array.random.random(shape, chunks=chunks)),
        "W": (dims, dask.array.random.random(shape, chunks=chunks)),
        "T": (dims, dask.array.random.random(shape, chunks=chunks)),
    },
    coords={"time": pd.date_range("2001-01-01", periods=shape[0], freq="H")},
)
zonal_means = ds.mean("lon")
anomaly = ds - zonal_means

anomaly['uv'] = anomaly.U*anomaly.V
anomaly['vt'] = anomaly.V*anomaly.T
anomaly['uw'] = anomaly.U*anomaly.W

temdiags = zonal_means.merge(anomaly[['uv','vt','uw']].mean("lon"))

# note method="blockwise" uses flox
temdiags = temdiags.resample(time="D").mean(method="blockwise")

wait(temdiags.persist())

I'd be curious to learn if this is necessary network traffic (in which case we should improve our handling of network) or if it's unnecessary network traffic (in which case we should improve our task placement).
In either case it seems like a reasonable and easy benchmark to add.  With 50 workers it looks like it'll take me 10 minutes or so (although this should probably decrease once we swap in the new hardware defaults).  We can tune this down easily if we want to.
If I were to ask for an improvement on this it would be to run on a real dataset rather than something random.

Well, not entirely network bound I guess.  It's oddly balanced between network and CPU.

Those seem too "easy" at a quick glance.

There are a couple of different overlapping goals here:

Benchmarks so that Dask development is sensitive to Pangeo community needs over time
Examples to show what's possible

These are different goals, but our hope is that these benchmarks/workflows can serve both.  We hope that by making things more demo-able that that'll force us more towards what people actually do, and away from artifical examples that highlight a particular technical challenge (There is probably an analogy here between playing sports and targetting specific muscle groups while working out at a gym.  We're looking for sports).
What I like about sea surface altitude is that the results are interpretable and pretty.  It also makes us sensitive to issues in Zarr, reading from cloud storage, s3 flakiness, etc..
@dcherian a lot of your recent examples are great because they're easy to run, but they lack some of the breadth of the real-world examples.  Thoughts?
I've only looked through the last few comments. I'm going to start working my way up the thread now.

Well, not entirely network bound I guess. It's oddly balanced between network and CPU.

I suspect a fair amount of the ~20% system cpu util is network-related, so most likely network bound and network also explains higher CPU util (but also CPU util is in there range where I don't think CPU is bottleneck)

Also FWIW I probably disagree a little with @jrbourbeau about external dependencies.  Mostly we want to make sure that the workload looks familiar to Pangeo users and represents their needs well.  If tools like xGCM or xhistogram or flox aren't likely to bias our thinking then I don't really care about whether or not their active.  If we find ourselves focusing a lot of engineering time one what ended up being an xGCM or xhistogram issue though then I think that we will have missed.

@ntabris https://cloud.coiled.io/clusters/192751/details?account=dask-engineering if you want to go deeper
(please anyone else let me know if you want access to Coiled to look at metrics.  It's easy to add you all to a team where we can look at this stuff together, or where you can run things)

What I like about sea surface altitude is that the results are interpretable and pretty. It also makes us sensitive to issues in Zarr, reading from cloud storage, s3 flakiness, etc..

@dcherian a lot of your recent examples are great because they're easy to run, but they lack some of the breadth of the real-world examples. Thoughts?

These are all based on real-world use cases but with random data swapped in for actual datasets. I can add those pieces back if you want "end-to-end" demos with pretty pictures.

https://github.com/coiled/benchmarks/issues/770#issuecomment-1506216575 stems from a user comment on xarray's issue tracker, but the data layout is like the "LLC" simulation that Tom/Julius are talking about.
The second one is the ERA5 dataset (quite famous!). This is a real workflow but skips a bit where the ERA5 dataset gets regridded to a coarser model grid for comparison purposes. I can work with the original author to make some nice images.
The third climatological mean is also ERA5, that one does actually hit the real dataset.

If I take https://github.com/coiled/benchmarks/issues/770#issuecomment-1506254519 from @dcherian above and swap out 20 years for 1 year it seems not terrible.

Also on this one I noted a large increase in head node memory use before it started computing.

These are all based on real-world use cases but with random data swapped in for actual datasets. I can add those pieces back if you want "end-to-end" demos with pretty pictures.

Pretty pictures and also we get to see the impact of reading real data, which I suspect will be significant.  For example, maybe there's some GIL-holding thing happening in computation which is affecting fsspec's ability to read data.  We want to see those kinds of interactions.


break scratch apple catcher steps up to be easier

Happy to do this, too
My thoughts:

Step 2 should be two parts (move left, move right)
Step 6 maybe could be more explicit about where the apple should go when you catch it? Or maybe that's too leading
Step 7 should be two parts (make the score variable, change the score)
Step 8 should be two parts (make the level variable, change the level)

Cool! I think someone already did this and we forgot to close the issue. However, looking over it now, there are some small things to change:

I think Step 1 could just instruct you to add one character and a backdrop and leave it to step 3 to pick the second falling sprite.

Agreed you could split up Step 2

Step 3 should include the words "When the flag is clicked"

Step 4 should indicate when it should fall

Agreed that Step 6 could be more explicit.

I don't know if we need to break up step 7. I like it as one step. Creating a variable and giving it a name doesn't feel like enough.

I don't know if we need to break up step 8. However, we could be more explicit that when the score is 3, we go to the next level and the score resets to 0.


Some notes from my first run of the script.

The script is very cool, especially how it does everything including the folder structure and the zipping!
These are notes about a run without --cn_unet.

If the file name has a . in it besides for the extension, like stable-diff-1.5.ckpt, will the split command still split it at the right place?  Does it look for the last . or the first . or does it split at all of them?  It would be fine to have a rule when you explain the script to someone that files names can only have a . at the extension, if that helps keep the code simple.

The xxxxx_original_512x512.zip needs to drop the _512x512.  If it is not in a subfolder with a size like 512x768, the CoreML repo says that means the model is a default 512x512.  The same way a split_einsum model does not get a size in the file name, or a 512x512 subfolder inside of the split-einsum folder.

I think the script should delete working folders as it completes each zip, and delete the xxxx_diffusers folder at the end of everything.  To use up less space on the working drive while it is running.

I will do a run with --cn_unet tomorrow night.  I think that one will need some minor changes, but I want to actually run it because my code reading skills are not that good.

I asked ChatGPT about the split('.') issue:
👱
Please explain what this line of python code does:  fileNameArr = fileName.split('.')
🤖
This line of Python code takes a string variable fileName and splits it into an array of strings using the period (.) as the separator. The resulting array is assigned to the variable fileNameArr. This is useful when you need to extract the file extension from a file name, as the last element of the array will be the extension.
👱
The code:  fileName = fileNameArr[0]  is used to define the fileName.  If the file name includes a . besides the extension separator . does this code use the parts of the file name up to the . at the extension, or only the part of the file name before a first . is encountered?
🤖
The line of code fileName = fileNameArr[0] takes the first element of the fileNameArr array and assigns it to the fileName variable. Therefore, if the file name includes a period (".") besides the extension separator, this code will only use the part of the file name before the first period (".") is encountered as the file name.
For example, if the original fileName variable is "my.file.name.txt", the split() method will create an array ['my', 'file', 'name', 'txt']. The line fileName = fileNameArr[0] will assign the first element of the array, "my", to the fileName variable, effectively ignoring the rest of the file name.
👱
How would I tell the code to combine all of the elements in the array back into the fileName, just omitting the last array element which is the file extension?
🤖
To combine all of the elements in the array back into the fileName, except for the last element which is the file extension, you can use the join() method to join the elements of the fileNameArr array, except for the last one, with a period separator. Here's an example code snippet:
# assuming fileNameArr is already defined
fileName = ".".join(fileNameArr[:-1])

In this code, fileNameArr[:-1] selects all the elements of the fileNameArr array except for the last element (which contains the file extension). The join() method then joins these elements together using a period separator to create the fileName.

Thank you for your notes. I'm glad to discuss this script.

I asked ChatGPT about the split('.') issue:
👱 Please explain what this line of python code does: fileNameArr = fileName.split('.')
🤖 This line of Python code takes a string variable fileName and splits it into an array of strings using the period (.) as the separator. The resulting array is assigned to the variable fileNameArr. This is useful when you need to extract the file extension from a file name, as the last element of the array will be the extension.
👱 The code: fileName = fileNameArr[0] is used to define the fileName. If the file name includes a . besides the extension separator . does this code use the parts of the file name up to the . at the extension, or only the part of the file name before a first . is encountered?
🤖 The line of code fileName = fileNameArr[0] takes the first element of the fileNameArr array and assigns it to the fileName variable. Therefore, if the file name includes a period (".") besides the extension separator, this code will only use the part of the file name before the first period (".") is encountered as the file name.
For example, if the original fileName variable is "my.file.name.txt", the split() method will create an array ['my', 'file', 'name', 'txt']. The line fileName = fileNameArr[0] will assign the first element of the array, "my", to the fileName variable, effectively ignoring the rest of the file name.
👱 How would I tell the code to combine all of the elements in the array back into the fileName, just omitting the last array element which is the file extension?
🤖 To combine all of the elements in the array back into the fileName, except for the last element which is the file extension, you can use the join() method to join the elements of the fileNameArr array, except for the last one, with a period separator. Here's an example code snippet:
# assuming fileNameArr is already defined
fileName = ".".join(fileNameArr[:-1])

In this code, fileNameArr[:-1] selects all the elements of the fileNameArr array except for the last element (which contains the file extension). The join() method then joins these elements together using a period separator to create the fileName.

I also ask ChatGPT just now. Your method is correct.

I should retain the working folders, the model can use in Mochi diffusion.I usually test the converted model after it has finished converting.

Yes, you are right.  It makes sense to have it to test.

These are my notes from the run with --cn_unet added to the command arguments.  I hope they make sense.
With the -cn_unet argument:
After it completes filling the xxxx_split-or-original_size_cn folder, it needs to copy the Unet.mlmodelc file from the resources folder of the xxxx_split-or-original_size folder to the resources folder of the xxxx_split-or-original_size_cn folder.
The xxxx_split-or-original_size_cn resources folder then has BOTH Unet types.  That resources folder is the one that needs to be zipped, with _cn added to the end of the file name before the .zip extension.  And then all of the _cn.zip files are used to build the folder tree, with all the folders also having _cn added to their names.  None of the resource folders with only the regular Unet get saved.  They were just there to create a regular Unet file that could be added to the other set.
The way you have it running twice is the only to get both Unets.  So don't change anything there.  Just at the end, copy the ome Unet type to the resources folder that has the other Unet type, and then zip the 7 pieces into the finished model, with _cn added in the name and _cn added to the names of the folders they get saved to, so we know it is a model that can work with ControlNet.
The logic is that the script makes either a final complete model with the regular Unet or a final complete model with both the regular Unet and the ControlledUnet.  It never zips a model with only the ControlledUnet type.  I don't think people would want that type.  They either use a model that doesn't need to do ControlNet, or a model that works with and without a ControlNet.

If the above set of comments made sense, here is a trick to make that logic simpler to execute.
When the script does a second run for the --cu_unet, it can save all the work to the same destination folder as the first run.  All of the files that are duplicated in a second run are exact duplicates of the first run files, and they can and will just overwrite them.  The first run's Unet.mlmodelc stays as it was, and the second run's ControledUnet.mlmodelc is an added file to the others.  Reusing the destination folder saves needing to move that one Unet from one set to the other, and it saves about half of the holding space needed while the two sets of files are collecting for the final zip file. The files and folders still need to all have _cn added somehow.

For this --cn_unet stuff, I think it means you need to define 2 different zip routines.
The first run alone never calls a zip routine.
if second run (is invoked with the --cn_unet)
call a zip routine that appends _cn to files and folders,
else
call a zip routine that doe not append _cn to files and folders (using the output from the first run)

"That's good. I am also trying to find a way to combine the two models: the normal model and the CN model. I will read your comments carefully. We can first convert the two models and then try to find a way to combine them.”


PA2558_Group5_fix string duplication for components a to j

Breaking change

Proposed change

Type of change

[ ] Dependency upgrade
[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New integration (thank you!)
[ ] New feature (which adds functionality to an existing integration)
[ ] Deprecation (breaking change to happen in the future)
[ ] Breaking change (fix/feature causing existing functionality to break)
[ ] Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #
This PR is related to issue:
Link to documentation pull request:

Checklist

[ ] The code change is tested and works locally.
[ ] Local tests pass. Your PR cannot be merged unless tests pass
[ ] There is no commented out code in this PR.
[ ] I have followed the development checklist
[ ] The code has been formatted using Black (black --fast homeassistant tests)
[ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly.
Updated and included derived files by running: python3 -m script.hassfest.
[ ] New or updated dependencies have been added to requirements_all.txt.
Updated by running python3 -m script.gen_requirements_all.
[ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
[ ] Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

[ ] No score or internal
[ ] 🥈 Silver
[ ] 🥇 Gold
[ ] 🏆 Platinum

To help with the load of incoming pull requests:

[ ] I have reviewed two other open pull requests in this repository.

When the program has duplicate code, it becomes lengthy and bulky. In order for your software to last a long time, you must have high-quality code. Code that is duplicated smells bad and increases technical debt. Eliminate duplicate code blocks, but the goal should be to reduce as much technical debt as possible. Your code will be of higher quality as a result. If you have used string duplication, it takes your time, efficiency, and sometimes even your temper to fix every location that code is located in.  copy and paste programming is more of an issue than duplicate code, but as with most instances of repetitive code generated through copy-and-paste, it deserves a mention. Plagiarized code has holes and endings that are not visible when someone takes a code from somewhere and adds it to their software. In the case of duplicate codes, attackers can exploit these openings and gain access to your code, making it vulnerable to attack.

Overall, looks good and is a reasonable refactoring.

Actually, there PR has introduced some syntax errors (see pylint and mypi CI results). Hence closing without merge.


Error on 'npm start'

I'm getting errors when I run npm start:
[kris@kris-pc delivery-link]$ npm start

><EMAIL_ADDRESS>start /run/media/kris/storage/delivery-link
> npm stop ; npm run truffle:migrate-ganache && HOST_IP=$(ip -4 addr show $(docker network inspect bridge --format='{{(index .IPAM.Config 0).Gateway}}') | grep -Po 'inet \K[\d.]+') docker-compose up -d && npm run truffle:init-setup

><EMAIL_ADDRESS>stop /run/media/kris/storage/delivery-link
> docker-compose down

WARNING: The HOST_IP variable is not set. Defaulting to a blank string.
ERROR: Couldn't find env file: /run/media/kris/storage/delivery-link/truffle/build/addrs.env
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>stop: `docker-compose down`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>stop script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kris/.npm/_logs/2019-11-21T19_49_35_340Z-debug.log

><EMAIL_ADDRESS>truffle:migrate-ganache /run/media/kris/storage/delivery-link
> cd ./truffle/ && npx truffle migrate --network ganache && cd ..

Compiling your contracts...
===========================
Error: Error: Error: Could not find chainlink/contracts/ChainlinkClient.sol from any sources; imported from /run/media/kris/storage/delivery-link/truffle/contracts/DeliveryLink.sol
    at Object.compile (/home/kris/.nvm/versions/node/v10.12.0/lib/node_modules/truffle/build/webpack:/packages/truffle-workflow-compile/index.js:103:1)
Truffle v5.0.24 (core: 5.0.24)
Node v10.12.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>truffle:migrate-ganache: `cd ./truffle/ && npx truffle migrate --network ganache && cd ..`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>truffle:migrate-ganache script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?

npm ERR!  Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kris/.npm/_logs/2019-11-21T19_49_52_379Z-debug.log

Hi, Kris - thanks for checking out this repo! I think you may be experiencing this problem because you either did not run the npm install command or it did not succeed. From the logs you provided, I can tell that the error is occurring when your Solidity contracts are being compiled, and in particular the error is that the compiler is not able to locate the Chainlink client smart contract dependencies; these dependencies should be pulled down when you run npm install. The only way I am able to reproduce this error is by trying to start the project without installing the dependencies first.

Ah, I think that was it haha. Maybe that first step in 'Usage' can be in the 'Dependencies' section to avoid ambiguity? Just an idea.

Hmm, actually, it might be something else. I ran npm i and still get the same error.  I did notice though that the Ganache window (after creating the new Workspace) hangs on the loading screen (the Ganache logo with spinning circle thing).
Running Ganache on command line, I notice this warning:
(node:16058) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 2 server-started listeners added. Use emitter.setMaxListeners() to increase limit

Hm, I am not sure about the problems with Ganache, but running npm i should take care of the errors you shared above. Can you share the output you're getting after running npm i? Whether or not Ganache is running shouldn't impact whether or not you can compile the contracts, which is what the error you shared above is telling me.

hey, I appreciate your taking the time to help me with this!
So after completing npm i, I get this when I run npm start with no Ganache running:
><EMAIL_ADDRESS>start /run/media/kris/storage/delivery-link
> npm stop ; npm run truffle:migrate-ganache && HOST_IP=$(ip -4 addr show $(docker network inspect bridge --format='{{(index .IPAM.Config 0).Gateway}}') | grep -Po 'inet \K[\d.]+') docker-compose up -d && npm run truffle:init-setup

><EMAIL_ADDRESS>stop /run/media/kris/storage/delivery-link
> docker-compose down

WARNING: The HOST_IP variable is not set. Defaulting to a blank string.
ERROR: Couldn't find env file: /run/media/kris/storage/delivery-link/truffle/build/addrs.env
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>stop: `docker-compose down`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>stop script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kris/.npm/_logs/2019-11-25T22_06_00_709Z-debug.log

><EMAIL_ADDRESS>truffle:migrate-ganache /run/media/kris/storage/delivery-link
> cd ./truffle/ && npx truffle migrate --network ganache && cd ..

Compiling your contracts...
===========================
> Compiling ./contracts/DeliveryLink.sol
> Compiling ./contracts/LinkToken.sol
> Compiling ./contracts/Migrations.sol
> Compiling ./contracts/Oracle.sol
> Compiling chainlink/contracts/Chainlink.sol
> Compiling chainlink/contracts/ChainlinkClient.sol
> Compiling chainlink/contracts/Oracle.sol
> Compiling chainlink/contracts/interfaces/ChainlinkRequestInterface.sol
> Compiling chainlink/contracts/interfaces/ENSInterface.sol
> Compiling chainlink/contracts/interfaces/LinkTokenInterface.sol
> Compiling chainlink/contracts/interfaces/OracleInterface.sol
> Compiling chainlink/contracts/interfaces/PointerInterface.sol
> Compiling chainlink/contracts/vendor/Buffer.sol
> Compiling chainlink/contracts/vendor/CBOR.sol
> Compiling chainlink/contracts/vendor/ENSResolver.sol
> Compiling chainlink/contracts/vendor/Ownable.sol
> Compiling chainlink/contracts/vendor/SafeMath.sol
> Compiling link_token/contracts/ERC677Token.sol
> Compiling link_token/contracts/LinkToken.sol
> Compiling link_token/contracts/math/linkSafeMath.sol
> Compiling link_token/contracts/token/ERC677.sol
> Compiling link_token/contracts/token/ERC677Receiver.sol
> Compiling link_token/contracts/token/linkBasicToken.sol
> Compiling link_token/contracts/token/linkERC20.sol
> Compiling link_token/contracts/token/linkERC20Basic.sol
> Compiling link_token/contracts/token/linkStandardToken.sol

    > compilation warnings encountered:

link_token/contracts/token/linkStandardToken.sol:27:5: Warning: Use of the "var" keyword is deprecated.
    var _allowance = allowed[_from][msg.sender];
    ^------------^
,link_token/contracts/LinkToken.sol:15:3: Warning: Defining constructors as functions with the same name as the contract is deprecated. Use "constructor(...) { ... }" instead.
  function LinkToken()
  ^ (Relevant source part starts here and spans across multiple lines).
,/run/media/kris/storage/delivery-link/truffle/contracts/DeliveryLink.sol:13:3: Warning: Variable is shadowed in inline assembly by an instruction of the same name
  uint256 public timestamp;
  ^----------------------^
,link_token/contracts/ERC677Token.sol:21:5: Warning: Invoking events without "emit" prefix is deprecated.
    Transfer(msg.sender, _to, _value, _data);
    ^--------------------------------------^
,link_token/contracts/token/linkBasicToken.sol:25:5: Warning: Invoking events without "emit" prefix is deprecated.
    Transfer(msg.sender, _to, _value);
    ^-------------------------------^
,link_token/contracts/token/linkStandardToken.sol:35:5: Warning: Invoking events without "emit" prefix is deprecated.
    Transfer(_from, _to, _value);
    ^--------------------------^
,link_token/contracts/token/linkStandardToken.sol:46:5: Warning: Invoking events without "emit" prefix is deprecated.
    Approval(msg.sender, _spender, _value);
    ^------------------------------------^
,link_token/contracts/token/linkStandardToken.sol:69:5: Warning: Invoking events without "emit" prefix is deprecated.
    Approval(msg.sender, _spender, allowed[msg.sender][_spender]);
    ^-----------------------------------------------------------^
,link_token/contracts/token/linkStandardToken.sol:81:5: Warning: Invoking events without "emit" prefix is deprecated.
    Approval(msg.sender, _spender, allowed[msg.sender][_spender]);
    ^-----------------------------------------------------------^
,link_token/contracts/token/linkERC20Basic.sol:11:3: Warning: No visibility specified. Defaulting to "public".
  function balanceOf(address who) constant returns (uint256);
  ^---------------------------------------------------------^
,link_token/contracts/token/linkERC20Basic.sol:12:3: Warning: No visibility specified. Defaulting to "public".
  function transfer(address to, uint256 value) returns (bool);
  ^----------------------------------------------------------^
,link_token/contracts/token/linkERC20.sol:12:3: Warning: No visibility specified. Defaulting to "public".
  function allowance(address owner, address spender) constant returns (uint256);
  ^----------------------------------------------------------------------------^
,link_token/contracts/token/linkERC20.sol:13:3: Warning: No visibility specified. Defaulting to "public".
  function transferFrom(address from, address to, uint256 value) returns (bool);
  ^----------------------------------------------------------------------------^
,link_token/contracts/token/linkERC20.sol:14:3: Warning: No visibility specified. Defaulting to "public".
  function approve(address spender, uint256 value) returns (bool);
  ^--------------------------------------------------------------^
,link_token/contracts/token/ERC677.sol:6:3: Warning: No visibility specified. Defaulting to "public".
  function transferAndCall(address to, uint value, bytes data) returns (bool success);
  ^----------------------------------------------------------------------------------^
,link_token/contracts/token/ERC677Receiver.sol:5:3: Warning: No visibility specified. Defaulting to "public".
  function onTokenTransfer(address _sender, uint _value, bytes _data);
  ^------------------------------------------------------------------^
,link_token/contracts/token/linkBasicToken.sol:22:3: Warning: No visibility specified. Defaulting to "public".
  function transfer(address _to, uint256 _value) returns (bool) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/token/linkBasicToken.sol:34:3: Warning: No visibility specified. Defaulting to "public".
  function balanceOf(address _owner) constant returns (uint256 balance) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/token/linkStandardToken.sol:26:3: Warning: No visibility specified. Defaulting to "public".
  function transferFrom(address _from, address _to, uint256 _value) returns (bool) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/token/linkStandardToken.sol:44:3: Warning: No visibility specified. Defaulting to "public".
  function approve(address _spender, uint256 _value) returns (bool) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/token/linkStandardToken.sol:56:3: Warning: No visibility specified. Defaulting to "public".
  function allowance(address _owner, address _spender) constant returns (uint256 remaining) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/token/linkStandardToken.sol:66:3: Warning: No visibility specified. Defaulting to "public".
  function increaseApproval (address _spender, uint _addedValue)
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/token/linkStandardToken.sol:73:3: Warning: No visibility specified. Defaulting to "public".
  function decreaseApproval (address _spender, uint _subtractedValue)
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/ERC677Token.sol:38:3: Warning: Function state mutability can be restricted to view
  function isContract(address _addr)
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/math/linkSafeMath.sol:9:3: Warning: Function state mutability can be restricted to pure
  function mul(uint256 a, uint256 b) internal constant returns (uint256) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/math/linkSafeMath.sol:15:3: Warning: Function state mutability can be restricted to pure
  function div(uint256 a, uint256 b) internal constant returns (uint256) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/math/linkSafeMath.sol:22:3: Warning: Function state mutability can be restricted to pure
  function sub(uint256 a, uint256 b) internal constant returns (uint256) {
  ^ (Relevant source part starts here and spans across multiple lines).
,link_token/contracts/math/linkSafeMath.sol:27:3: Warning: Function state mutability can be restricted to pure
  function add(uint256 a, uint256 b) internal constant returns (uint256) {
  ^ (Relevant source part starts here and spans across multiple lines).

> Artifacts written to /run/media/kris/storage/delivery-link/truffle/build/contracts
> Compiled successfully using:
   - solc: 0.4.24+commit.e67f0147.Emscripten.clang

> Something went wrong while attempting to connect to the network. Check your network configuration.

Could not connect to your Ethereum client with the following parameters:
    - host       > <IP_ADDRESS>
    - port       > 7545
    - network_id > *
Please check that your Ethereum client:
    - is running
    - is accepting RPC connections (i.e., "--rpc" option is used in geth)
    - is accessible over the network
    - is properly configured in your Truffle configuration file (truffle-config.js)

Truffle v5.0.43 (core: 5.0.43)
Node v10.12.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>truffle:migrate-ganache: `cd ./truffle/ && npx truffle migrate --network ganache && cd ..`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>truffle:migrate-ganache script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!  Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kris/.npm/_logs/2019-11-25T22_06_21_432Z-debug.log

Totally happy to help and really appreciate the same from you :grin: The error you shared above makes perfect sense if Ganache is not running. What is happening is that the contracts compiled successfully and then they are attempting to be deployed to the local Ganache network. Since that is not running, you're getting this error: Something went wrong while attempting to connect to the network. Check your network configuration. I don't really have any experience with the Ganache CLI and the GUI has always "just worked" for me :persevere:

At the workshop at the brewery, I noticed that the Ganache GUI you were using was different than the one I was using, and realized I was using an old version (which didn't have that workspace setup on startup). So I upgraded to the version you were using, but I may just try the CLI for now.

Don't forget to get it to listen on all interfaces! (<IP_ADDRESS>)

running this to start ganache-cli:
ganache-cli -h <IP_ADDRESS> -p 7545

looks like it compiled and migrated, but still giving an error. looks like it's docker related:
https://gist.github.com/krisc/8da0dae2cdac03e1c39bc410a7083783

Not sure if I should do this, but maybe this is helpful. So I ran these series of commands:
ganache-cli -h <IP_ADDRESS> -p 7545
sudo dockerd
duso npm start

And it went a little further along, but still got an error:
https://gist.github.com/krisc/fe448b8bbe4cb3dc95fd57a7aef64566

Are you running on Mac or Linux? This seems to point to HOST_IP env var not getting set.
WARNING: The HOST_IP variable is not set. Defaulting to a blank string.
This seems be a problem on Mac as the command ip will not work. This bug is particularly pervasive as it only shows as a warning, but the actual effect is that the Chainlink node is unable to connect to Ganache.
If you are a Mac user, I'd suggest replacing in package.json:
HOST_IP=$(ip -4 addr show docker0 | grep -Po 'inet \\K[\\d.]+')
With:
HOST_IP=$(ipconfig getifaddr en0)
This seems to be the only Linux OS-specific bug in the workshop as Docker will work on any platform. Tbh, my solution is also kinda a OSX-specific hack. I think it would be better practice if  HOST_IP was set as an environment variable.

@leovigna  hmm yeah, we overlooked that WARNING. But I'm actually on Manjaro Linux ...

So did you find a way to set HOST_IP? Are the docker instances able to connect?

Hi @leovigna, I have the same issue
'''
Using network 'ganache'.
Transfering 100 LINK to 0xB62EA3D281c48BFd6A0D058Dc905Acf585BF697a...
Transfer succeeded! Transaction ID: 0xfd2a501c9dc674137de9ab62f0db25b84eaec1deea3c4a723e2939b9365b26a1.

<EMAIL_ADDRESS>truffle:prep-node /Users/mv/Documents/Workspace/develop/delivery-Link
cd ./truffle && npx truffle exec ./scripts/prep-node.js --network ganache && cd ..

Using network 'ganache'.
/Users/mv/Documents/Workspace/develop/delivery-Link/chainlink/cl-utils.js:54
resolve(JSON.parse(resBody).data[0].id);
^
SyntaxError: Unexpected end of JSON input
at JSON.parse ()
at IncomingMessage. (/Users/mv/Documents/Workspace/develop/delivery-Link/chainlink/cl-utils.js:54:22)
at IncomingMessage.emit (events.js:323:22)
at endReadableNT (_stream_readable.js:1204:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
I've also got the warning for not setting the HOST_IP but when I echo it does show the IP. Also from the video from the workshop, you can see this is the same behavior as when Dan ran it, so not sure this is the issue.
Seems it has an issue with 'cl-utils.js' and as can be seen above fails in the prep-node.js script
can this be related to web3 version?
Thanks!

Hi @leovigna, I have the same issue
'''
Using network 'ganache'.
Transfering 100 LINK to 0xB62EA3D281c48BFd6A0D058Dc905Acf585BF697a...
Transfer succeeded! Transaction ID: 0xfd2a501c9dc674137de9ab62f0db25b84eaec1deea3c4a723e2939b9365b26a1.
<EMAIL_ADDRESS>truffle:prep-node /Users/mv/Documents/Workspace/develop/delivery-Link
cd ./truffle && npx truffle exec ./scripts/prep-node.js --network ganache && cd ..
'''
Using network 'ganache'.
/Users/mv/Documents/Workspace/develop/delivery-Link/chainlink/cl-utils.js:54
resolve(JSON.parse(resBody).data[0].id);
^
SyntaxError: Unexpected end of JSON input
at JSON.parse ()
at IncomingMessage. (/Users/mv/Documents/Workspace/develop/delivery-Link/chainlink/cl-utils.js:54:22)
at IncomingMessage.emit (events.js:323:22)
at endReadableNT (_stream_readable.js:1204:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
I've also got the warning for not setting the HOST_IP but when I echo it does show the IP. Also from the video from the workshop, you can see this is the same behavior as when Dan ran it, so not sure this is the issue.
Seems it has an issue with 'cl-utils.js' and as can be seen above fails in the prep-node.js script
can this be related to web3 version?
Thanks!

Hey, I've struggled to set the whole thing locally as well, which it has not been as "plug & play" as in the video (but in fairness nothing never is).
My approach has been a bit bottom up after GORM wasn't able to connect to my local PostgreSQL instance (not containarised) on he first attempt using this repository.
A summary of what I've done is (for local development purposes):

Install Docker Engine, Docker Compose and be sure they work for not root users.
Install PostgreSQL 12 and configure both postgresql.conf and pg_hba.conf for being able to be reached from a container.
Set up the database user, database, etc.
Follow the Chainlink official documentation for just spinning up a node.
Go back to this repository and try to understand better the whole integration.

Now I'm blocked at the same point than you: prep-node.js fails at clUtils.getAcctAddr(), because this function is unable to send the GET request (http://localhost:6688/v2/user/balances). I've isolated the request in a minimal Truffle script and it does not work either. The funny thing is the request works both via browser and Postman.
Call using Truffle
Shell script call:
npx truffle exec scripts/smoke_test.js

Shell response (socket hangs because there is no response):
Using network 'ganache'.

Error: socket hang up
    at connResetException (internal/errors.js:609:14)
    at Socket.socketOnEnd (_http_client.js:453:23)
    at Socket.emit (events.js:327:22)
    at endReadableNT (_stream_readable.js:1221:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

Call using Postman
{
    "data": [
        {
            "type": "accountBalances",
            "id": "0xda280e62e9C8c60D90dDA58a49928d5a2B1002b2",
            "attributes": {
                "address": "0xda280e62e9C8c60D90dDA58a49928d5a2B1002b2",
                "ethBalance": "0",
                "linkBalance": "0"
            }
        }
    ]
}

I will keep investigating what is going on. Any help appreciated @danforbes, otherwise I will ask at Discord.
Thanks

Morning, quick follow up on the matter!
I was wrong and the function to blame is getSessionCookie(). I'm running Chainlink Node 0.8.12 (because it wasn't locked in the docker-compose.yml), and with this version the statement const cookie = res.headers["set-cookie"][0] assigns a wrong cookie. There are two items in the set-cookie Array and the one that matters starts with clsession=. As I can't assure their positions my quick fix is:
        res.on("end", () => {
          const clsessionIdx = res.headers["set-cookie"].findIndex((c) =>
            c.startsWith("clsession")
          );
          cookie = res.headers["set-cookie"][clsessionIdx];
          resolve(cookie);
        });

@Mvilsht hope it helps, but probably you have already sorted it out.

Sorry for the very delayed reply, @vnavascues. I'm glad you were able to make progress on this. Since this repo clearly isn't being maintained. I am going to archive it.


Every second, a new mpv instance starts

Your OS: Windows 11
Node.JS version: v20.10.0
NPM version: 10.2.3
Problem description:
When I start mpv, every second, a new mpv instance starts.

What's your MPV version?

v0.37.0-139-gab5b2503

I've tried with  v0.37.0-135-gabc2a748 (yesterday build) and Node.JS v21.5.0 it works fine for me.
Try to update to latest version of mpv.
If it doesn't work, how do you launch MPV?
Are you using the latest version of mpv-remote-app and mpv-remote-node?

It was caused by option in mpv.conf:
input-ipc-server = /tmp/mpvsocket

Thanks for letting me know!

Thanks for the support.


Add paybackLoan dftx + tx builder

What kind of PR is this?:

/kind feature
What this PR does / why we need it:
Which issue(s) does this PR fixes?:

Fixes part of #532
Additional comments?:

loanPayback => paybackloan Refactor accordingly sweat_smile

sure.. will do... because the ain side still remain loanPayback on raw side.. that's the reason i didn't rename it

loanPayback => paybackloan Refactor accordingly sweat_smile

sure.. will do... because the ain side still remain loanPayback on raw side.. that's the reason i didn't rename it

just saw that ain side still use loanPayback as message name.
Now i'm not sure. @fuxingloh any comment?

loanPayback => paybackloan Refactor accordingly sweat_smile

sure.. will do... because the ain side still remain loanPayback on raw side.. that's the reason i didn't rename it

just saw that ain side still use loanPayback as message name. Now i'm not sure. @fuxingloh any comment?

Yeah, I think we can and should strive to keep the consistency on our side.

Similar stance on whale engineering, if it has nothing to do with consensus and purely for naming purposes, we do our best to keep to our own consistency and paradigm.


Add the code coverage for this repo

Current Behavior

I do not have test coverage for the code, it is hard for us to review and merge the PR with the same code quality. And we want
to increase the code quality. So, we need to add coverage to the repo.
Desired Behavior

Implementation

Acceptance Tests

Mockups

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Wonderful

This issue is related to PR #468

Closing by PR #471


Unknown encoder 'libfaac'

Q
A

Bug?
perhaps

New Feature?
no

Version Used
[0.7.0] - 2016-12-15

FFmpeg Version
FFmpeg  3.2.2-1~bpo8+1

OS
Debian 8.6

Actual Behavior
I try to pad a clip and I get a PHP Fatal Error.
Expected Behavior
No crash 😁
Steps to Reproduce
Use the padding filer on a video file.
Possible Solutions
After a manual execution of the command I saw the actual error: Unknown encoder 'libfaac' .
As I see here they removed this codec from ffmpeg: http://stackoverflow.com/questions/19774975/unknown-encoder-libfaac
Posible solution: use simple aac

Problem 2: I just want to add padding to a video to get it to a bigger resolution.
/usr/bin/ffmpeg '-y' '-i' '/tmp/rec3.2.mp4' '-vf' 'scale=iw*min(1366/iw\,768/ih):ih*min(1366/iw\,768/ih),pad=1366:768:(1366-iw)/2:(768-ih)/2' '-vcodec' 'libx264' '-acodec' 'aac' '-refs' '6' '-coder' '1' '-sc_threshold' '40' '-flags' '+loop' '-me_range' '16' '-subq' '7' '-i_qfactor' '0.71' '-qcomp' '0.6' '-qdiff' '4' '-trellis' '1' '-b:a' '128k' '-pass' '1' '-passlogfile' '/tmp/ffmpeg-passes5898cc0144c73hoobv/pass-5898cc0144d30' '/tmp/rec3.2_pad.mp4'
ffmpeg version 3.2.2-1~bpo8+1 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: ......
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/rec3.2.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
encoder         : Lavf57.56.100
Duration: 00:00:05.73, start: 0.000000, bitrate: 605 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 473 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Metadata:
handler_name    : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name    : SoundHandler
[libx264 @ 0x7f4719907600] using SAR=1/1
[libx264 @ 0x7f4719907600] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x7f4719907600] ratecontrol_init: can't open stats file
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))`
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

I found that the problems is because of the parameter -pass 1 . I don't know what this is does but I'm going to find a way to remove from the command.

Ok. There were other errors like typos in the output filename or the folder for pass was not existent when I was running the command manually. After all, the only problem was with the codec and I used the fix from issue #295


Added Hindi translation

Added hindi translation to CONTRIBUTE docs

Please review my pull request and merge it.

This changes not required, please pick the issue having Hacktoberfest label on it  and make a valid contribution. thanks

Sure, thanks.


Re-enable touch screen functions

Hi Prof,  great work as always.  I have just updated to the new opencore setup and have successfully upgraded to 10.15. 6
Is there a way to reenable the touch screen whist keeping the track pad improvements? The track pad is working much better with multitouch but I use the touchscreen frequently in applications such as logic and other VST softwares for music production and it would be great to have the option to use it again

Hi @R-Teer , try the TPXX.aml attached file: I re-enabled touchscreen mode with GPI0 pinning mode -> so it should work better! You just need to replace the old one in OC/ACPI folder.
Let me know your feedback please!
N.B. I disabled touchscreen to satisfy this request: I'm sorry for the late reply!
SSDT-TPXX.aml.zip

Works fantastic man! Much appreciated!

@profzei  Hi prof greate job for keeping this repo so useful
I'm a bit of newbie here may i ask where does SSDT file does? how to use it?
many thanks

An SSDT file is only an hot patch applied to original DSDT.aml for our machine... If you are using this repo, you only need to put it in your /EFI/OC/ACPI folder and active it in your /EFI/OC/config.plist
A very good starting point is the Dortania Team Guide

@profzei Thanks man for a clear answer, I was trying to use SSDT-TPXX.aml by enable SSDT-TPXX-NO.aml while still enable SSDT-TPXX.aml and the result is trackpad and screen did not response.
Also I used a different way is to delete SSDT-TPXX.aml and rename SSDT-TPXX-NO to SSDT-TPXX also trackpad and screen were not responsing any ideals how to fix it?
Thanks in advance, and so sorry my english is not good

@LavenderMP Also my English is not good...
Try the following:
SSDT-TPXX.aml.zip
Put it in EFI/OC/ACPI folder and replace the existing file with the same name


adjust ACEs for acs-commons-package-replication-status-event-service to

also treat packages with ACEs correctly
Improved exception handling as well
This closes #1327

Coverage increased (+0.006%) to 38.618% when pulling d50609fcba1bc3698ee49b8d5fc47f18a4882b68 on kwin:bugfix/1327_replication-status-updater into 77c558b7de9c90ba02f2d9163c7dc1b0ace95248 on Adobe-Consulting-Services:master.

Manually merged to remove unrelated import changes.


[cinder-csi-plugin]  Update debian base image to v2.1.3

What this PR does / why we need it:
Update debian base image to v2.1.3 to address CVEs.
Release note:

NONE

/ok-to-test

/test cloud-provider-openstack-acceptance-test-csi-cinder

/test cloud-provider-openstack-acceptance-test-csi-cinder

@ialidzhikov looks like its unable to find the version 2.1.3 , https://logs.openlabtesting.org/logs/80/1180/68a6e132902bde8c056a5ed1f4bc96b2f9c3d26e/cloud-provider-openstack-acceptance-test-csi-cinder/cloud-provider-openstack-acceptance-test-csi-cinder/d7c92f6/job-output.json.gz

@ramineni , @jichenjc , sorry I had a typo in the base image. It should be good with the latest force push - ref  https://github.com/kubernetes/cloud-provider-openstack/compare/68a6e132902bde8c056a5ed1f4bc96b2f9c3d26e..95108bc09bcfd463b3ee3298543f775ccc6656c6.

why not only update with last version directly instead of change repo?

@jichenjc , k8s.io.gcr is a vanity domain that will resolve to {eu,us,...}.gcr.io/k8s-artifacts-prod/. The last time we did similar base image update (ref https://github.com/kubernetes/cloud-provider-openstack/pull/1077), I believe the vanity domain flip was in process and it was possible to use only the {eu,us,...}.gcr.io/k8s-artifacts-prod/ images.

@ialidzhikov thanks for the info , the upgrade is reasonable to me ~
/lgtm

/lgtm
/approve


Create minecraft

Insert Pack Name(s) Here
Minecraft Theme
Insert a link to the GitHub repository of the pack.
https://github.com/Jmotz85/Minecraft-Theme
Checklist
Failure to complete this checklist or deleting boxes from the checklist will result in the closing of your pull request unless this is a pack update. Please write any comments regarding this checklist at the bottom of your pull request.
Check every box.

[x ] I am the original author of this pack or have permission from the original author to make this pull request.
[x ] All copyright of this pack's contents belong to the listed author or is cited in the description and repository linked above.
[x ] This theme works properly on the latest versions of SteamOS for Steam Deck, decky-loader and SDH-AudioLoader.
[ x] This pack is under 10MB in size and uses the least disk space possible.
[ x] This pack is safe for work and does not contain any sexual, drug-related, or profane content.
[ x] This pack does not contain loud, distorted, or otherwise bad faith sound files.
[ x] This pack is only for either music or sounds and only adds or modifies the intended sound files.

This is my first try uploading a pack, I may need some help lol


[GLUTEN-1434] [VL] Optimize shuffle get batch

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
(Fixes: #ISSUE-ID)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

The function from will combine a 1 column ColumnarBatch(pid) and a several columns Columnarbatch(data), if the latter batch columns are too much, this will be a burden. The old function addColumn use std::move(children) to avoid generating all new children vector. @zhztheplayer

if the latter batch columns are too much, this will be a burden.

Ideally it just brings some meta-data level copy-and-pastes out of the data. Is there some real performance regressions observed from your side?

No, it may gain performance in user situation.

Would you like to rebase? Thanks.

I optimize the code logic to remove removePid, the old logic is too complicated.

For velox shuffle, we don't need Arrow schema_, do you think we should move to velox types? @zhztheplayer There may be a lot of code refactor.

For velox shuffle, we don't need Arrow schema_, do you think we should move to velox types? @zhztheplayer There may be a lot of code refactor.

By this did you mean using the current approach that infers schema from the first columnar batch?


Fix Performance for Large HTML Documents (Wikipedia)

If I put a large HTML document in (ie Wikipedia) my app freezes, removing the parseDOM fixes it. I don't need rerendering - I just need one pass, so I added a prop to improve performance - now my app is slick! This is perfect for chat apps etc.
I made it a flag as I see if you want people to be able to dynamically change the style it will help but to honest - you should use react-fast-compare and StyleSheet.flatten all styles and diff them for the parseDOM operation - take this HTML doc - https://en.wikipedia.org/wiki/Xixiasaurus

@evanjmg You can control the re-rendering by making sure the props passed to HTML component are equal between two rendering cycles of the parent component. It seems that your PR offers an option to freeze the rendering while there are more obvious reasons to deal with re-rendering issues. But if you think I missed something, I'd be glad to hear from you.


Migrate flutter/infra to git-on-borg

Currently the source of truth is https://github.com/flutter/infra and is mirrored to git-on-borg. Instead, we'd like to move a CL flow to enable automation to submit changes to the repo.

[x] Create builder to validate generated configs (equivalent to the cirrus check) - https://flutter-review.googlesource.com/c/recipes/+/14920
[x] Send announcement about freeze to github source
[ ] ~Nest flutter/infra in an infra/ folder to make it easier to support mono-repo~
[x] Switch source of truth to git-on-borg (chrome-internal luci configs) - go/chrome-internal/3944716
[x] Archive github repo

This looks mostly read to go, we're CPing one last change to catch GoB up to GitHub, then we can switch over. I'll send PSA after this has been switched as it's a low traffic repo.

I got temporary access as flutter-gob-owner and was able to remove the no longer necessary write acls. Now waiting on LUCI to update the source of truth.
flutter/infra has been marked archived, with a redirect in the readme to the gob version.

PSA was sent, LUCI CL landed, and i'm now waiting for it to propagate before closing this.

This is complete. LUCI only imports new shas if they change the luci configs. infra HEAD has two commits which did not make any relevant changes (TIL!)


Drop .env file support and keep only TOML configuration file

Currently we support both that this results in extra work and an higher chance to have stale .example files with can result in a worse UX. Also TOML is more flexible compared to an .env file.

#316 is actually doing the opposite and that PR motivates why.


feat(SearchBar): Categorize search results in search bar

Thank you for your contribution to the KodaDot NFT gallery.
👇 _ Let's make a quick check before the contribution.
PR type

[ ] Bugfix
[x] Feature
[ ] Refactoring

What's new?

[x] PR closes #2376
[x] Categorize search results in search bar

Before submitting Pull Request, please make sure:

[x] My contribution builds clean without any errors or warnings
[x] I've merged recent default branch -- main and I've no conflicts
[x] I've tried to respect high code quality standards
[x] I've didn't break any original functionality
[x] I've posted a screenshot of demonstrated change in this PR

Optional

[ ] I've tested it at </rmrk/collection/26902bc2f7c20c546a-1FVG7>
[x] I've tested PR on mobile and everything seems works
[ ] I found edge cases
[ ] I've written some unit tests 🧪

Had issue bounty label?

[x] Fill up your KSM address: Payout

Community participation

[x] Are you at KodaDot Discord?

Screenshot

[x] My fix has changed something on UI; a screenshot is best to understand changes for others.

https://user-images.githubusercontent.com/38389586/155739763-b35b3cc0-50e1-4dd2-9a39-e1c79a4e88d8.mp4

Let's put collections first as people look for them more often, items later.
I will test once done :)

Let's put collections first as people look for them more often, items later. I will test once done :)

How about "History" and "Search"?
Do we still need them?

Let's put collections first as people look for them more often, items later. I will test once done :)

How about "History" and "Search"?
Do we still need them?

Good q. History should be kept but history searching 🔎 term is already in upper row. So it could be removed :)
Will check once I'm around on 💻

done~

works!

pay 100 usd

😍 Perfect, I’ve sent the payout
💵 $100 @ 133.02 USD/KSM ~ 0.752 $KSM
🧗 EgCzZLpVe7zg7RHh79qPyC8UrykVkA22YY2oj7sWMZf15Ws
🔗 0xee44b6ba3efc3ed8d09e42e72ac7eab636b1c30147cd2e9ecc4a734ddcf40a21
🪅 Let’s grab another issue and get rewarded!
🪄 github.com/kodadot/nft-gallery/issues


cannot use Xbox Elite Series 2 controller under Arch Linux even after using custom mappings

Describe the bug
Even after using cusom mappings, I cannot use my controller on Arch at all since it cannot find the mappings. I also use xpadneo and xone.
To Reproduce
Steps to reproduce the behavior:

Install xpadneo driver and connect controller
create a custom mapping
Start the mod
see that the name of the controller is yellow and nothing works

Expected behavior
I expected the controller to be functional
Desktop (please complete the following information):

OS: Arch Linux (6.0.11-arch1-1-t2)
Minecraft [1.19.0]
Fabric [fabric-0.14.9]
Mods [MidnightControls, Fabric API]
Version [1.7.0-1.19]
Branch [main]

Additional context
I have both the xone and xpadneo drivers installed for my controller so i can use the wired mode too. I have not seen any issues or incompatabilites in journalctl around it.

Please enable debug mode in the Advanced Config, restart the game with the controller plugged in, close it, and send me the log.

Same issue here with an Xbox One S pad over USB2.
Reproduction steps (assuming environment):

Launch SDL2 Gamepad Mapper (aur package)
Make a mapping.
Load using MidnightControls.
Reload mappings.
Try to bind anything.

Expected behavior:
Bindings work without any issues. Controls work in-game.
Actual behavior:
Only LT and RT are placed on a bind. Control does nothing in-game.
Environment:

Arch Linux, kernel 6.1.4-arch1-1
xpad driver
Minecraft 1.19.2
MidnightControls 1.7.4-1.19.3
Fabric 0.14.12, Fabric API 0.72.0
Modrinth release.

Worth noting that in the mapper tool and another game, the controller works as intended, so it's likely something with LWJGL or MidnightControls itself.

restart the game with the controller plugged in, close it, and send me the log.

Logs are available here.

Try this mapping string:
030000005e040000ea0200000f050000,Xbox One S Controller,a:b0,b:b1,x:b2,y:b3,back:b6,guide:b8,start:b7,leftshoulder:b4,rightshoulder:b5,leftstick:b9,rightstick:b10,dpup:h0.1,dpleft:h0.8,dpdown:h0.4,dpright:h0.2,-leftx:-a0,+leftx:+a0,-lefty:-a1,+lefty:+a1,-rightx:-a3,+rightx:+a3,righty:a4,lefttrigger:a2,righttrigger:a5,platform:Linux,

See #89


Foundations - Typography

Used well typography helps to enhance readability, establish hierarchy and set a visual tone, resulting in more effective communication.
### Tasks
- [x] Create initial draft
- [x] Complete draft
- [x] Request review
- [x] Receive feedback from DS team
- [x] Address internal feedback
- [x] Request review from DXD designers
- [x] Receive feedback from DXD designers
- [x] Address DXD feedback
- [ ] Ready to publish to Salt site

Review tomorrow by design, progress continuing

@parsakhan21 i currently have a blocker on this. We need to be able to flag content within the salt site as ‘in progress’ or ‘planned/in backlog’ in order for us to release partially completed content and elicit feedback. The enhancement is currently with James and Joanna

Designs for this are currently in Figma.


Discontinuities after detach

At the moment, we emit discontinuity events after attaching channels after explicitly detaching them.
Should we forgo these events if the channel has been detached explicitly?
┆Issue is synchronized with this Jira Task by Unito

We emit reason:? ErrorInfo as a part of discontinuity.
Why not include the reason as explicitly detached while emitting discontinuity event, wdyt

At the same time, I would argue not to expose a lot of internal stuff,  where customers need to add extra check for discontinuity error type.
It would make sense to emit discontinuity when it actually happens 👍  ( when there was no explicit user intevention )


pool swap - allow editing of token to swap

What would you like to be added:
Allow editing of token to swap
Why is this needed:

Acceptance criteria
Expected: Ability to edit Token B, currently only able to edit input of Token A
Android:

[ ] 1. DEX > Select dETH-DFI > Swap > input 1 dETH (Token A), should reflect 1,XXX DFI (Token) (Token B) > change input of Token B
[ ] 2. || > swap token position > input 1 DFI (Token A), should reflect 0.00000990 dETH (Token B) > change input of Token B
[ ] 3. || > Select dBTC-DFI > ||
[ ] 4. || > Select dUSDT-DFI > ||
[ ] 5. || > Select dDOGE-DFI > ||
[ ] 6. || > Select dLTC-DFI > ||
[ ] 7. || > Select dBCH-DFI > ||
[ ] 8. || > Select dUSDC-DFI > ||

iOS:

[ ] 1. DEX > Select dETH-DFI > Swap > input 1 dETH (Token A), should reflect 1,XXX DFI (Token) (Token B) > change input of Token B
[ ] 2. || > swap token position > input 1 DFI (Token A), should reflect 0.00000990 dETH (Token B) > change input of Token B
[ ] 3. || > Select dBTC-DFI > ||
[ ] 4. || > Select dUSDT-DFI > ||
[ ] 5. || > Select dDOGE-DFI > ||
[ ] 6. || > Select dLTC-DFI > ||
[ ] 7. || > Select dBCH-DFI > ||
[ ] 8. || > Select dUSDC-DFI > ||


Revert "Added a object structure for GroupingRequest objects, accessable from…"

Reverts vespa-engine/vespa#6512
@bratseth PR I suspect this one broke grouping, we must investigate after next week.
@henrhoi FYI

👍


Replace gsl::at with a new til::at(span) for pre-checked bounds

NOTE :x: This merges into another topic branch -- I'll retarget and merge to master after the topic lands.
The recent changes to use gsl::span everywhere added a few bounds checks
along codepaths where we were already checking bounds. Some of them may
be non-obvious to the optimizer, so we can now use til::at to help them
along.
To accomplish this, I've added a new overload of til::at that takes a
span and directly accesses its backing buffer.

depends on #6921

This PR is absolutely horked thanks to its base branch being deleted. Recovering.


python3: Add test-management to py3-backend and gracefully give up in email-mirror

Running ./manage.py email-mirror used to fail on python 3 because twisted.mail.imap4 is not python 3 compatible. Display a message informing the user that email-mirror is not available on python 3 instead of failing with a traceback.
Also add tools/test-management to py3-backend.

Automated message from Dropbox CLA bot
@sharmaeklavya2, it looks like you've already signed the Dropbox CLA. Thanks!

Merged, thanks @sharmaeklavya2!


Refactor button styles:- hover transitions repeated & fixed border-radius

I have refactored the button:hover styles where the transitions were repeated as they are already in the base button style.
Changed the border-radius from % to rems and this rounds the corners nicely without the sharp edges.
I have added the cursor:pointer property so the cursor changes to a hand when hovering over the button.

Great, thanks for the changes. Merging now.


Mandatory Boolean can only be set to "true"

Hi,
I tried your 30-day evaluation schema form generator with a schema of mine, and discovered what I believe to be a bug. If minOccurs for a type="boolean" is set to 1, your program only lets you set it to "true", otherwise the tag is removed and you cannot save the file.
In addition, I'm not totally convinced that that an invalid file cannot be saved is the best design decision. Besides that, I think your form generator is nice!

Hi!
Thanks for checking it out.
Yes, it's true that required booleans (either through minOccurs="1" or use="required") have to be checked in order to be valid. This mimicks the 'accept terms and conditions' behavior of checkboxes.
I am not sure if the specification says something about the behavior of required booleans. A simple test with https://www.freeformatter.com/xml-validator-xsd.html showed that required booleans both allow true and false, but they do have to be declared. Perhaps my implementation is too strict...
Saving invalid XML files kind of goes against the purpose of the application... ;) Do you have a use scenario where you'd like to be able to do that?

Hi, I looked into the documentation and 'false' booleans are indeed valid when required. I updated the source code accordingly. Booleans now can take both values. I'll shortly update the XML Form Generator installer as well.

Just checking in to say I've released a new version with several features and bug fixes, including the boolean bug you mentioned. Both the GitHub version and the trial download have been updated.


Can't get started - igvR() fails

Hi all,
I'm just trying to get started using this tutorial.
After installing from bioconductor, I get errors immediately while trying to use igvR(). With verbose reporting on, I see:
> igv <- igvR(quiet = FALSE)
want to load
BrowserViz constructor starting with html file ''
 html file exists? FALSE
Error in BrowserViz(portRange, title, browserFile = igvBrowserFile, quiet,  :
  file.exists(browserFile) is not TRUE

It sounds like there needs to be some sort of html file available that igvR can't create? Would appreciate some guidance!

Whoops, this might be important too: Ubuntu 18.04
> sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
 [1] AnnotationHub_2.14.5        igvR_1.2.0                  BrowserViz_2.4.0            httpuv_1.5.0                jsonlite_1.6
 [6] rtracklayer_1.42.2          VariantAnnotation_1.28.13   Rsamtools_1.34.1            Biostrings_2.50.2           XVector_0.22.0
[11] SummarizedExperiment_1.12.0 DelayedArray_0.8.0          BiocParallel_1.16.6         matrixStats_0.54.0          Biobase_2.42.0
[16] GenomicRanges_1.34.0        GenomeInfoDb_1.18.2         IRanges_2.16.0              S4Vectors_0.20.1            BiocGenerics_0.28.0

loaded via a namespace (and not attached):
 [1] progress_1.2.0                lattice_0.20-38               htmltools_0.3.6               yaml_2.2.0
 [5] GenomicFeatures_1.34.7        interactiveDisplayBase_1.20.0 blob_1.1.1                    XML_3.98-1.19
 [9] rlang_0.3.2                   later_0.8.0                   DBI_<IP_ADDRESS>01                bit64_0.9-7
[13] GenomeInfoDbData_1.2.0        stringr_1.4.0                 zlibbioc_1.28.0               memoise_1.1.0
[17] biomaRt_2.38.0                AnnotationDbi_1.44.0          Rcpp_1.0.1                    xtable_1.8-3
[21] promises_1.0.1                BSgenome_1.50.0               BiocManager_1.30.4            mime_0.6
[25] bit_1.1-14                    hms_0.4.2                     digest_0.6.18                 stringi_1.4.3
[29] shiny_1.2.0                   grid_3.5.3                    tools_3.5.3                   bitops_1.0-6
[33] magrittr_1.5                  RCurl_1.95-4.12               RSQLite_2.1.1                 crayon_1.3.4
[37] pkgconfig_2.0.2               Matrix_1.2-17                 prettyunits_1.0.2             assertthat_0.2.1
[41] httr_1.4.0                    rstudioapi_0.10               R6_2.4.0                      GenomicAlignments_1.18.1
[45] compiler_3.5.3

Hi @paul-shannon @taytayp ,
I got the same error here. I found the igvBrowserFile was defined in the .onLoad.
igvBrowserFile <<- system.file(package="igvR", "browserCode", "dist", "igvApp.html")

But the browserCode directory in the version 1.2.0 of igvR from @Bioconductor is empty.
I tried to install the github version, which contains this file.
devtools::install_github("paul-shannon/BrowserViz")
devtools::install_github("paul-shannon/igvR")

However, I got another error to start igvR.
> igv <- igvR()
attempting to open websocket connection on port 15000
Error in as.vector(x, "character") :
cannot coerce type 'environment' to vector of type 'character'

Thanks for your help!

@hubentu good call, glad it's not just me.
I get the same error when loading the github code. Searching through the repo, looks like as.vector(x, "character") doesn't show up. Sounds like igvR is passing an environment when it's not supposed to.

@taytayp @hubentu  Thanks for the bug reports - most helpful.   I'll have a fix by the end of the day.

@taytayp @hubentu Could you try a fresh pull and then R CMD INSTALL .?  I had updated the bioconductor with my changes over the last many weeks, but not this repo.   It is current now, package version 1.3.4
Note that the distribution includes code to build igvApp.html from javascript source - but you do not need to do that.  igvApp.html, already assembled, is in the repo, and ends up in the package, and is loaded by igvR.R in this preamble code:
.onLoad <- function(...){
   igvBrowserFile <<- system.file(package="igvR", "browserCode", "dist", "igvApp.html")
   }

Let me know if you have any problems - I'll get right to them as you report them.   My apologies for not keeping my own repo up to date.

@paul-shannon No worries, thanks for the speedy updates.
I'm still getting the same environment issues. My steps:
git clone https://github.com/paul-shannon/igvR.git
cd igvR
R CMD INSTALL .
R
library(igvR)
igvR()

I'm not missing anything here, am I?

@taytayp
Here's what I get, following the same steps - though retaining and then making method calls on the igvR object returned by the constructor:

Hi @paul-shannon ,
I got the same error as @taytayp . My BrowserViz (2.5.7) and httpuv (1.5.0) versions are different from your session.
Could you also update the BrowserViz package in your github? I guess the error might come from this package.
Thanks!
> igv = igvR()
attempting to open websocket connection on port 15000
createTcpServer: address already in use
attempting to open websocket connection on port 15001
Error in as.vector(x, "character") :
  cannot coerce type 'environment' to vector of type 'character'
> sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets
[8] methods   base

other attached packages:
 [1] igvR_1.3.4                  BrowserViz_2.5.7
 [3] httpuv_1.5.0                jsonlite_1.6
 [5] rtracklayer_1.42.2          VariantAnnotation_1.28.13
 [7] Rsamtools_1.34.1            Biostrings_2.50.2
 [9] XVector_0.22.0              SummarizedExperiment_1.12.0
[11] DelayedArray_0.8.0          BiocParallel_1.16.6
[13] matrixStats_0.54.0          Biobase_2.42.0
[15] GenomicRanges_1.34.0        GenomeInfoDb_1.18.2
[17] IRanges_2.16.0              S4Vectors_0.20.1
[19] BiocGenerics_0.28.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1               later_0.8.0              compiler_3.5.3
 [4] prettyunits_1.0.2        GenomicFeatures_1.34.7   bitops_1.0-6
 [7] tools_3.5.3              zlibbioc_1.28.0          progress_1.2.0
[10] biomaRt_2.38.0           digest_0.6.18            bit_1.1-14
[13] BSgenome_1.50.0          RSQLite_2.1.1            memoise_1.1.0
[16] lattice_0.20-38          pkgconfig_2.0.2          rlang_0.3.2
[19] Matrix_1.2-17            DBI_1.0.0                GenomeInfoDbData_1.2.0
[22] httr_1.4.0               stringr_1.4.0            hms_0.4.2
[25] bit64_0.9-7              grid_3.5.3               R6_2.4.0
[28] AnnotationDbi_1.44.0     XML_3.98-1.19            magrittr_1.5
[31] blob_1.1.1               promises_1.0.1           GenomicAlignments_1.18.1
[34] assertthat_0.2.1         stringi_1.4.3            RCurl_1.95-4.12
[37] crayon_1.3.4

@paul-shannon @hubentu
It looks like we're both on R 3.5.3 while Paul is on 3.5.1. I'm going to try and install the older version to see if that's the culprit. It seems unlikely, but those little differences can make a big deal at times.

R versions are less likely to explain the problem than @hubentu’s suggestion that I failed to push updates to BrowserViz.  I will remedy that now - let’s see how that goes.
Thanks to you both for your patience!

Paul

On Mar 29, 2019, at 11:29 AM, Taylor<EMAIL_ADDRESS>wrote:
@paul-shannon @hubentu
It looks like we're both on R 3.5.3 while Paul is on 3.5.1. I'm going to try and install the older version to see if that's the culprit. It seems unlikely, but those little differences can make a big deal at times.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

After upgrading to httpuv 1.5.0, I now get the error you report:
Error in as.vector(x, "character") :
 cannot coerce type 'environment' to vector of type 'character'

I'll do my best to fix this by the end of the day.   Thanks again!

@hubentu @taytayp  It appears that lots has changed with httpuv 1.5.0.  I am working through them now.  I will keep you posted.

@hubentu @taytayp
igvR version 1.3.6  and BrowserViz version 2.5.12 now accomodate httupuv 1.5.0
Let me know how it goes for you after pulling, installing both packages, then running igvR

Working great for me! Thanks for your quick support, Paul. I'll let you know if any other issues come up. I'm excited to work with the package.

Also works great for me! Thank you for your quick fix.


CB-10849: Improve error message for manual FreeIPA Azure backup

Improve the error message for manual FreeIPA Azure backup. When a
user runs the Azure FreeIPA backup inside a sudo environment which
doesn't replace the keyctl session then it fails due to access
insufficient access.
This was tested with a local deployment on Azure.
See detailed description in the commit message.

Looks good @jamisonbennett.  With these changes for Azure, do we need to make improvements for AWS and GCP?

Looks good @jamisonbennett.  With these changes for Azure, do we need to make improvements for AWS and GCP?

@jasonwmcswain Only Azure CLI uses the keyctl service to access the keyrings. No changes are needed for the other cloud providers.

@jasonwmcswain Only Azure CLI uses the keyctl service to access the keyrings. No changes are needed for the other cloud providers.


Possible mistake in op2 link data

Hello, our team is trying to upgrade framework for our ancient OP2. When studying the code, we found the hardcoded parameters quite different from what we measured. Is there anything wrong with these position parameters?
I guess they are for OP3, rather than OP2. Is this right?
https://github.com/ROBOTIS-GIT/ROBOTIS-OP2/blob/9e18ae6e1b3cc9a770a0823872300274e520caf9/op2_kinematics_dynamics/src/op2_kinematics_dynamics.cpp#L348
op2_link_data_[13]->relative_position_ = robotis_framework::getTransitionXYZ(0.000, 0.000, -0.110);
// just like this one, we think this should be (0.0, 0.0, -93.0)

Thank you for your time.

Hi @ridethepig
The actual length of the link of the z axis is -0.093 (in meters) as you correctly measured.
It seems those links are back ported and left unchanged from the source code of OP3.
https://github.com/ROBOTIS-GIT/ROBOTIS-OP3/blob/master/op3_kinematics_dynamics/src/op3_kinematics_dynamics.cpp#L334
Unfortunately, the OP2 is discontinued, therefore, further internal test will not be performed to fix this discrepancy.
Thank you.

That's really unfortunate.
Well, we have to try fixing these by ourselves.


Upgrade Home Assistant Docker image to version 2024.11.3

This pull request updates the Home Assistant Docker image to version 2024.11.3.

coverage: 99.901%. remained the same
when pulling 9a7bb69c207bb702e38619dc10b6fc8163b067d0 on upgrate_home_assistant_docker_image
into 49afa4b4a676761de4ba989a8dc2921eb8e1f415 on master.


Add support for cancellation tokens

Hello!
There is a pattern for comfortable and safe stopping of the functions - cancellation tokens. I have created an implementation of it - cantok. I suggest to use it in the readkey function like in this example:
import readchar
import cantok

token = cantok.TimeoutToken(5)
key = readchar.readkey(token=token) # by default the token is just SimpleToken()

Into the readkey function you need to read the state of the token periodically. If it was cancelled - you need to stop read the stdin and return something or raise an exception.
Notice, my suggestion is very similar to an another about non-blocking reading of the stdin, but it is more universal.

Thank you for the suggestion.
Using tokens seems overkill though, a simple timeout is probably sufficient. This can be implemented at the moment though, as readchar internaly gets blocked by kernel calls, which makes impossible to perform any form of checks (be it time or token based).
See the development branch, where a more versatile version is under development, which would allow for this kind of implementation. It would also allow the user to implement his own token based variant instead of the default timeout version, if he wishes. It not fully working though and I didn't find the time to continue on it.

@Cube707 Could you say only for me - now is there a way to stop reading of a char except to send the char to stdin? From the inside of a program.

At the moment the only way is to spawn a thread and kill that, to send a interrupt to the process or to force readchar to finish consuming input.

to spawn a thread and kill that, to send a interrupt to the process

How can i interrupt the thread?

see #43 for a simple example. after you create the thread and store its management object in a variable, you have methods available to kill it.

 I thought i tested this, but looks like I am wrong

Yes, looks like i can't do it. This is a problem for me. Now i make some program where i need to read some chars and automatically stop it when the program is stopped. I do not see any way to do it.

If the program is terminated it will stop (CTRL+C, CTRL+D or any other form of SIG).
This libary was created for textual user interfaces and alike. There you are sure you can block, as only a input from the user requires you to continue. Everything else is currently out of scope.
For games or similar interactive things, you would need a more low-level input driver anyway.

also, using the multiprosessing libary and spawning a seperate process works:
from multiprocessing import Process
from time import sleep, time

from readchar import key, readkey

# construct an inverted code -> key-name mapping
# we need to revese the items so that aliases won't overrive the original name later on
known_keys = {v: k for k, v in reversed(vars(key).items()) if not k.startswith("__")}

def keylogger():
    print("starting keylogger...")
    while True:
        try:
            data = readkey()
        except KeyboardInterrupt:
            break
        if data in known_keys:
            data = known_keys[data]
        print(f"{data}-key was pressed")

if __name__ == "__main__":
    p = Process(name="keylogger", target=keylogger)
    p.start()
    sleep(5)
    print("timeout..")
    p.kill()


Final 0.4.0 release

When can we expect a final 0.4.0 release containing #50 & #52 on top of rc2?

Today


Add material thumbnail

Created materials have no thumbnail

This has to do with the appleseed bin path being erased when certain modules in the addon are updated.  Once you relink the bin directory the preview works.  Suggest closing this one.

This has to do with the appleseed bin path being erased when certain modules in the addon are updated.

Is there something we can do on our side?

I'm not sure.  I think it's a problem similar to the one with EnumProperties.

This one hasn't reared it's head in a while.  Can we close it?

This one hasn't reared it's head in a while. Can we close it?

I think so yes, especially since with the upcoming appleseed.python-based version of blenderseed, there's no longer any appleseed bin path setting, right?

Yep, it’lol all be bundled together.
Closing.


Add explicit Dynamic and Static product libraries

SwiftPM promises it can always correctly detect what library type (dynamic or static) should be use in the project. It is the case when all dependencies are given as a source code.
This is not always possible if some of the dependencies are binary targets. In that case SPM is not able to figure out what libraries are compiled (or linked) inside the binary. In that case SPM cannot properly resolve dependencies and set proper linking type for used libraries.
Currently there is also no easy way to list those dependencies for binary targets.
In this PR we propose to add explicit targets that provide required linking:

SwiftProtobufDynamic for dynamic linking
SwiftProtobufStatic for static linking

Those new targets will not break current setups and allow to manually manage dependencies if SPM cannot handle them properly by itself.

What is the reason for this? Normally, SPM should decide when a target should be linked statically or dynamically.

As I mentioned in the description. When working with binary targets SPM is not able to detect it correctly.
Let say we have a library which uses SwiftProtobuf. We don't publish the library code but instead publish compiled binaries on a public repository.
If you use the library in your own project SPM is not aware that the SwiftProtobuf was used. So if you also need SwiftProtobuf in your project you will end up with two same libraries. And depending what type of linking you choose (and what type was chosen for the library), you may or may not face some issues related with that.
In that case explicit specifying dynamic linking in both places solves the problem.

This is expected behaviour and SPM intentionally does not support this right now. If you bring in a binary dependency which requires specific dependency versions to be linked into the final executable then you are in all kinds of trouble. At this point SPM cannot help you anymore.
The current recommended practice is that either:

Your binary framework drops all its dependencies
Your binary framework starts to internalise all of its dependencies via something like git submodules

We shouldn't go around and add dynamic and static targets to all packages just because someone tried to ship a binary framework that depends on SPM packages. The real solution here would be make SPM aware of dependencies for binary targets and how they should be linked. This is something worth raising over in the SPM repository.

You can specify what dependencies (and in what versions) are needed to use your product. That's not a problem.
The main issue here is that linking type (static by default) is inbuilt into library and client cannot change it.
I assume that soon or later such functionality will be available in SPM. But now we have to play with what we have.

https://github.com/apple/swift-protobuf/issues/1066 and https://github.com/apple/swift-protobuf/pull/1043 also provide some context for this from the past.
@tbkka fyi

ps - I also worry that some other package will explicit pick to use the static or dynamic one, and thus end up with duplicate symbols because some other package picks different, so when both get included into a single binary, you get the code twice via these different targets.

That why separate target is proposed so you have to explicit chose it. By default SPM does its magic.
In this PR the dynamic target is most important. Static one is added only for completeness. We can remove it to not encourage others to use them.

ps - I also worry that some other package will explicit pick to use the static or dynamic one, and thus end up with duplicate symbols because some other package picks different, so when both get included into a single binary, you get the code twice via these different targets.

That why separate target is proposed so you have to explicit chose it. By default SPM does its magic.

I'm not sure I follow.  As soon as there is the option to pick, there is the risk different packages will pick different values, and then when those two packages end up in a single binary there are duplicate definitions.

I'm not sure I follow. As soon as there is the option to pick, there is the risk different packages will pick different values, and then when those two packages end up in a single binary there are duplicate definitions.

Right, there is such a risk. But without it we are forced to use static linking, ending up with increasing binary size and warnings that same method is available is multiple places and system will pick any of them randomly

Right, there is such a risk. But without it we are forced to use static linking, ending up with increasing binary size and warnings that same method is available is multiple places and system will pick any of them randomly

I think this is just a problem with how your dependencies are currently delivered to you. We cannot influence the choice that you have a framework which depends on swift-protobuf being linked into the final executable and there is sadly not much we can do here to help you work around this. As @thomasvl said we cannot offer two products with different linkage because this is going to diverge the ecosystem.
For your immediate help what you can do is just maintain a fork of swift-protobuf where you do that and ask whoever delivered you that framework to remove the protobuf dependency or internalise it.

Thanks for your input. I understand your concerns.
One more thing worth to mention. Please have a look at the official code snippets for SPM package template. Looks we are not the first ones that came with such idea.

The document you linked is merely documenting the APIs that SPM offers but doesn't recommend to do this at all. I still think the problem you are having is worth solving it just not up to individual packages to solve but rather for SPM to solve this. I would open an issue over in the SPM repository for dependencies for binary targets.

Thanks all of you for contribution. In this case we will try to solve the issue different way.
For now we can close the PR


Hide UIViewController method/properties in UIKit Example code

I noticed that a few ViewControllers in the UIKit example code had some methods and properties that could be hidden.
Cheers🙂

Got it, I'll keep that in mind👌Thanks


Fix showing trend graph

For both freestyle and pipeline builds.
I don't really know what I'm doing, I just copied https://github.com/jenkinsci/pmd-plugin/commit/3fda8ed3752bc5404f936c1e65ee9c37e3ea13b0

Just had my dev machine torpedoed, I'll check this and #16 over in the next day or two as I get things together again.


Submit an expense in different currency

@sedubois should be able to send his invoice in EUR to the OpenCollective collective which is in USD.
(same for me for my expenses in Europe)
We should add a dropdown in the form to submit an expense:

[ ] On website: https://github.com/OpenCollective/opencollective-website/blob/a45f891fc0a81bae1585e0b6b007460a328208c0/frontend/src/components/ExpenseForm.js
[ ] On app: https://github.com/OpenCollective/opencollective-app/blob/c14ba8d312b50600f6d5573da39393d24de8affa/frontend/src/components/TransactionForm.js

We should do this. We need to investigate how Paypal handles currency conversion - what data is reported to us and the currency of the transaction, etc. It would be a good opportunity to complete the migration to the new Expense model as well - it's more stable and scalable.

@asood123 @xdamman we already get the money in group currency remaining in preapprovalDetails response after submitting expense. The issue is that we just didn't store this amount. So to do:

[ ] store preapprovalDetails response (which includes amount remaining) in PaymentMethod.data
[ ] create script to populate these preapprovalDetails for all paypal PaymentMethods
[ ] update Transaction.netAmountInGroupCurrency based on subtraction from amount in preapprovalDetails response after paying expense, with previous value stored in PaymentMethod.data
[ ] also take opportunity to remove from codebase currencies (such as INR) not supported by PayPal
[ ] adapt app and website to specify currency in submit + edit expense forms

Other possibilities observed:

it's technically possible to hold several wallets for different currencies, instead of converting to a single one, but that would probably require more effort
instead of relying on preapprovalDetails data, we could rely on the doc which gives the rates (percentage + fixed) (but that's probably fragile) + call the ConvertCurrency API but it gives "an estimated exchange rate"

Been thinking about this for a few days. Main issue is that we aren't dealing with fees properly, so every solution creates more complication. We should accurately store the fees. It's fine to do it with another call afterwards (that's how we do it for stripe - we make a separate call after the txn finishes).
@xdamman do you know why we went with adaptive-payments, instead of REST API?

@asood123 found a solution today, always host currency and conversion on the fly. not sure when/how will be implemented. closing this.


http3: reject header field names with invalid characters

Depends on #3964.
cc @WeidiDeng 🙏

:shipit: