Re-write in C using WASM

Parsing the PGS files could be slow on low-end devices using JavaScript. I'm working on a C implementation using WebAssembly  with Emscripten and WebWorker support.

I'd advise being cautious about this one. Currently, WASM works flawlessly in Firefox (regarding raw performance), but in Chromium browsers it's really slow. You can do a quick test here with an intensive workload, it's night and day: https://ffmpegwasm.netlify.app/docs/getting-started/usage/
In my thesis I did quite a few tests with WASM and I came to the conclusion that WASM doesn't give noticeable improvements for performance-critical scenerios, in it's current state it's main purpose is for those applications that can't be ported/developed in JavaScript, like our libass in SubtitlesOctopus or Google Earth and Photoshop (for some public app examples).
If you want to do some benchmarks in a real application, you can install blurhash-wasm into our current web client or Jellyfin Vue and compare it with the current WebWorker implementation (there might be differences in the algorithm implementation, but I think it's one simple enough test that you could try)
What if you split up the current worker into 2 workers? One for OffscreenCanvas and another for the other? So they don't block each other. If it's still slow in some devices, you can still give it a go with memoization.

@ferferga Thanks for your advice.
  I spend the last week looking into and learning WebAssembly and discovered the big overhead WASM is adding to this project. The only area I saw an issue was allocating many Color instances when parsing the palette. Structs could solve this. However, most subtitles only use a hand full of colors. Also browsers are optimized to handle many instances in an array of the same type. For fun I'll try to store the pixel data in one big byte array and measure if it makes any difference.
I also got a Raspberry PI 4 to test. If I can view a 1080p video with the current PGS implementation on Chromium and Firefox without adding to much CPU load I'll call it a day for now.

Made some good changed in #9 to improve the performance.
It runs smoothly with 1080p - 10 Mbps on a Raspberry PI 4 (4gb, no overclocking) on Chromium.
Firefox already struggled with 1080p video streaming. Subtitle rendering or clearing can make the video lag. Pixel data encoding is not the issue though. clearRect and drawImage are blocking the browser - even from a web-worker. I guess the software renderer is just that slow.


Does typedJSON support typescript union types ?

Everything is in the title, I'm talking about | types not about discriminated unions

Hey @LifeIsStrange, this is not really possible at the moment. We are using reflect-metadata for getting the type of a property and currently it returns Object if you have a union type.
However, you may be able to do something depending on your case. You can add your own serialiser/deserialiser. For simple types (string|number|object) you can just use an identity function for that. For classes you could write an if statement (ex. on a property occurring on only one of the types) and recursively call TypedJSON. You can also configure a typeResolver that would resolve the actual type in a similar way; here you would also probably be interested to adjust the typeEmitter to not emit a type hint in the output.
Hope that helps

Hey @samstronghammer, sorry for such a delay. I have updated the documentation with a description and an example. You can also take a look at the tests to see a complete example.


iOS 8 - custom headers on a Manager instance are nil

I am creating a custom instance of Alamofire Manager and attaching additional headers to the configuration prior to initializing. I then use that manager to make a GET request. Here is how it is being initialized:
let config = NSURLSessionConfiguration.defaultSessionConfiguration()
config.timeoutIntervalForRequest = 20
config.TLSMinimumSupportedProtocol = .TLSProtocol12

var headers = config.HTTPAdditionalHeaders ?? [:]
headers[AppValues.NetworkValues.InternalOperationKey] = AppValues.NetworkValues.InternalOperationKey
headers["Accept-Encoding"] = "gzip"
headers["Accept"] = "application/json"

config.HTTPAdditionalHeaders = headers

manager = Manager(configuration: config)

Using this setup, on iOS 9, when I make the request using manager.request, the headers are attached properly and everything executes as expected. On iOS 8.4, executing the same code, the headers are empty on the request. If I attempt to print out the NSURLSessionConfiguration's additionalHTTPHeaders object using po manager.session.configuration.HTTPAdditionalHeaders prior to making the request, I can see the headers are populated. They are just not making it through to the actual request.
Am I performing setup incorrectly here?

Hi @ski081,
I can't spot anything obviously wrong in your sample code. I'd recommend checking out the tests we have verifying this behavior. If that doesn't help you spot your issue, then I'd suggest opening a question on Stack Overflow with more info.
Cheers. 🍻

From our Contribution Guidelines
Asking Questions
We don't use GitHub as a support forum. For any usage questions that are not specific to the project itself, please ask on Stack Overflow instead. By doing so, you'll be more likely to quickly solve your problem, and you'll allow anyone else with the same question to find the answer. This also allows maintainers to focus on improving the project for others.


Slicer one-click python dependencies

Dr. Andras Lasso offered to advise how to install dependencies with one-click. I am reaching out.

One important simplification would be to not require an additional Python environment, but use Slicer's Python environment for installing Pytorch and any other required dependencies. Usually we just pip_install what is needed, but Pytorch is a bit special (large package, must be installed from URL, etc.), so its installation is delegated to the Pytorch Slicer extension. See how it is done in TotalSegmentator extension - https://github.com/lassoan/SlicerTotalSegmentator/blob/main/TotalSegmentator/TotalSegmentator.py
Another simplification is to launch the processing server directly from Slicer and instead of zmq and memory-mapped files, send the pickled numpy array via process stdin/stdout. This technique is used in the Processes module in ParallelProcessing extension (https://github.com/pieper/SlicerParallelProcessing/blob/master/Processes/Processes.py) and it is very fast and robust and works on all operating systems. The module could be used as is (as shown in this example). If we find that there is a significant delay due to launching the process, importing pytorch, etc. then we could modify the implementation so that the server keeps running (does not terminate after processing a single image).

Creating a new branch https://github.com/bingogome/samm/tree/dev-py-dep-slicer


Add a license

@jahnique @webwurst any preferences which license to choose? How about MIT?

Resolved by #15


support any tags instead of only version and appID

Coverage decreased (-0.06%) to 61.458% when pulling 0de371fb8ae9e8b60195fc72f86fd449bff69bac on ninokop:master into 876f71416a465533a042db62eff0f541225c5b6f on ServiceComb:master.


Remove unneeded AjaxView usage

This PR removes an unneeded dependency on AjaxView in prep for https://github.com/inventree/InvenTree/pull/8384. Also updates the schema for the information endpoint and removes the general "not found" endpoint from the schema.

Nice addition, thanks @matmair

@SchrodingersGat slowly chipping away at CUI removal


Replace second h1 with h2 .title

Closes #6

@Tookmund I think #6 is not a problem, so I'm closing this pr. If there's more discussion on this issue, I'll reconsider.


Persona shutdown

"Persona is no longer actively developed by Mozilla. Mozilla has committed to operational and security support of the persona.org services until November 30th, 2016. On November 30th, 2016, Mozilla will shut down the persona.org services. Persona.org and related domains will be taken offline. If you run a website that relies on Persona, you need to implement an alternative login solution for your users before this date. For more information, see this guide to migrating your site away from Persona: https://wiki.mozilla.org/Identity/Persona_Shutdown_Guidelines_for_Reliers"
Minutes from call, http://wiki.geneontology.org/index.php/Ontology_meeting_2016-10-27#persona.org
Chris: you’ll see this reflected first in Noctua; will be able to log in via GH or Google; that’ll happen before the GOC mtg. For TG, we’ll probably use the same mechanism we use for Noctua. TG may have to be down for a week or so during that change.

Now dupe of geneontology/termgenie#98


Read and understand the entire task

The task is described in full in chapter 29 of the GPU Gems 3 textbook.
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch29.html

For the most part the task is understood. A few of the in-depth details and formula's need to be investigated further to make sure I understand why they are used.


error on copy & paste an image in markdown editor

Package
filament/filament
Package Version
v2.17.4
Laravel Version
v9.52.4
Livewire Version
v2.12.1
PHP Version
PHP 8.2.2
Problem description
There is an error on copy & paste an image in markdown editor, which happened in Chrome and Safari, but doesn't in Firefox.
This is a screenshot from Safari.

This is a screenshot from Chrome.

Expected behavior
It should work in Chrome and Safari, as it work in Firefox.
Steps to reproduce
copy & paste an image in markdown editor
Reproduction repository
https://github.com/simonwong1985hk/markdown-issue-filament
Relevant log output
No response

Probably introduced by https://github.com/filamentphp/filament/pull/5727.
Could you have a look, @awcodes @danharrin?


[Question] How to use this lib with "Remember me" feature

This library is the one I'm considering using for our application and one of the things I'm looking for among other things like using this with the magic link, custom reset password link, etc. is implementing a "Remember me" feature.
With the firebase/auth client, we can do this via session storage persistent, but with this lib, how would you achieve some similar functionality? It doesn't have to be exactly like that but my thought initially was not to have an expiration date since cookies without an expiration date will automatically be deleted as soon as the user closes the browser. The only issue is that expiration has to be defined as part of this lib's configuration. 💭

Hey @JeromeDeLeon,
Great question.
According to Web HTTP Cookie docs, if Set-Cookie header does not contain Max-Age or Expires attribute, it will expire when session ends.
You could achieve "Remember me" functionality by changing session cookie configuration based on some boolean value, as in following example:
export async function middleware(request: NextRequest) {
  const rememberMe = request.headers.get('x-remember-me') === 'true';

  return authMiddleware(request, {
    ...
    cookieSerializeOptions: {
      path: '/',
      httpOnly: true,
      secure: true, // Set this to true on HTTPS environments
      sameSite: 'lax' as const,
      maxAge: rememberMe ? 12 * 60 * 60 * 24 : undefined
    },
    ...
  });
}

I will close the issue now. Feel free to reopen if you have any further questions. Cheers 🎉


Align CowData data with its actual data instead of max_align_t.

This decreases alignment of all CowData on some systems from 16 bytes down to 8, decreasing memory use slightly.
For example, alignof(max_align_t) is 16 on macOS. The starting offset is guaranteed to be in alignment to uint64_t, due to the previous struct entry, which is 8. Since we don't use long long or long double, this means we can save 8 bytes for every CowData instance without breaking alignment.
Includes #100145.

It seems that compilation is failing because some structs are used in CowData before being defined, and MSVC cannot handle this.
I think we can make it work by reordering declarations and definitions such that Vector<X> is never used before X is defined.
Or we switch to use compiler-defined alignment by using fake structs instead of defining alignments manually. I'm open to both. Before I'm implementing anything new, I'd like to hear feedback though.

Both your solutions are not doable without some massive rewrite. I guess they wanted the flexibility to be able to define CowData<T> without having T defined yet, and you are breaking that here. Seems like lot of trouble for a very small gain.
I am curious though on how GCC and clang are compiling at this point, as they do not seem to in godbolt

Your godbolt example did not compile because it was using X in a function without any definition at all. You can see the equivalent example compiles equivalently on GodBolt:

https://godbolt.org/z/van944jKc (clang, success)
https://godbolt.org/z/f4hbEEnc3 (msvc, failure)

I guess they wanted the flexibility to be able to define CowData without having T defined yet

I'm not sure that is the case, but it's hard to know without asking the original author of this section of code, because it has not been commented. My guess was that this code was simply copied over from memory.h without looking at it in-depth.

Seems like lot of trouble for a very small gain

I guess that's true; i was hoping it would just compile everywhere, because then it would not have been any trouble at all. As is so often the case, much code was built around the code in its current state with its current setup implicitly assumed, making a change difficult.
I think my second solution (using structs instead of manual alignment) may still work though, without needing to touch any dependants.

Ok, I did some actual tests with a struct-based allocator.
Starting the launcher from b7c81d52bb988a218e19a3a199f5f2d11e4ff1a6 vs 25a0c9f5e750fe7c886c204dbe2b7f09cdb0b89e, I'm logging allocated RAM and boot times.
Boot times are about 0.67s +- 0.02s, negligible differences in both.
Allocation size is reproducibly 316149575b in b7c81d52bb988a218e19a3a199f5f2d11e4ff1a6, and 314894086 in 25a0c9f5e750fe7c886c204dbe2b7f09cdb0b89e. The compiler seems to prefer adding some padding somewhere; I'm not sure why.
In either case, it's not really worth risking a refactor for, or spending much more time on it. Therefore, I'm closing this PR in light of evidence.


Gate documentation overhaul

As part of making the gates into a circuit library...
Adding docstrings for the standard gates including matrix definition, some explanation of what they are, and what they look like in a circuit.
With this I remove the jupyter execute blocks in the documentation that were slowing things down (fixes #3906 )
I also added a RZX gate.
I mostly just changed docstrings so hopefully it won't clash with #3947 or what @Cryoris is doing with gates location.
I did have to change the matrix definition of RYY because the phase factor should not be part of the canonical matrix. For consistency all Pauli rotations are exp(-i.(\theta/2).pauli). And the decomposition is phase correct as well, so no extra phase is required. Only problem is our RZ to U1 unrolling loses the phase, which is something we should fix separately.

This creates merge conflicts in all the standard gate files in qiskit/extensions/standard since I'm working on a PR to move them to qiskit/circuit/gates. But since you only changes the docstrings this is easy to fix 👍


FlatList getting greyed out when using with LayoutAnimation

Description
FlatList getting greyed out when using with LayoutAnimation
PS. Seems an Android issue only
React Native version:
System:
OS: macOS 10.15.7
CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
Memory: 56.23 MB / 16.00 GB
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 14.2.0 - /var/folders/mf/rjybpq0x1d5f05hy4kmy4pz80000gn/T/yarn--1613012655622-0.7237216406452947/node
Yarn: 1.22.4 - /var/folders/mf/rjybpq0x1d5f05hy4kmy4pz80000gn/T/yarn--1613012655622-0.7237216406452947/yarn
npm: 6.14.4 - /usr/local/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
Managers:
CocoaPods: 1.10.0 - /usr/local/bin/pod
SDKs:
iOS SDK:
Platforms: iOS 14.1, DriverKit 19.0, macOS 10.15, tvOS 14.0, watchOS 7.0
Android SDK:
API Levels: 25, 26, 28, 29
Build Tools: 25.0.3, 28.0.3, 29.0.3
System Images: android-21 | Google APIs Intel x86 Atom_64, android-25 | Google APIs Intel x86 Atom, android-28 | Google Play Intel x86 Atom, android-29 | Google APIs Intel x86 Atom
Android NDK: Not Found
IDEs:
Android Studio: 4.1 AI-201.87<IP_ADDRESS>53283
Xcode: 12.1/12A7403 - /usr/bin/xcodebuild
Languages:
Java: 1.8.0_251 - /usr/bin/javac
Python: 2.7.16 - /usr/bin/python
npmPackages:
@react-native-community/cli: Not Found
react: 16.13.1 => 16.13.1
react-native: 0.63.4 => 0.63.4
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Steps To Reproduce

Create a populated FlatList (optionally rendered by state)
Use LayoutAnimation before making the FlatList visible as documented here
Move the items during the animation
Some items get greyed out

Expected Results
All items visible after the animation.
Snack, code example, screenshot, or link to a repository:
https://snack.expo.io/@gurzoni/issue-flatlist-vs-layoutanimation (Please try on Android)
https://user-images.githubusercontent.com/12538626/107599324-00837f00-6c85-11eb-85d3-fba72743fbcb.mp4

I'm having the same issue, some items show up with lesser opacity. Some items in rare cases don't even show up

@williamgurzoni @WouterVanHecke
I just remembered that there is a workaround for this issue. Not sure if it works for all cases but it works for the example.
Maybe this workaround can give a hint for the solution to solve this issue as well.
workaround is to set removeClippedSubviews={false} on flatlist. It is true by default for android and not ios for some reason.


Nested sections warning added section_tags.md

A recent change to MP has made it so nesting sections will no longer work. A warning has now been added to the docs to help prevent issues.

Hello @kylearoberts,Thanks again for the PR! It's HACKTOBERFEST! We want to show our appreciation by sending                                              you some special Hacktoberfest swag. If you have not already, could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


Try Astral uv for CI

author: @Andrei-Aksionov
Hello there 👋
This PR replace pip with blazingly fast uv by Astral.sh.
Couple of notes:

Decided to not use cache since uv is so fast, that the process of retrieving and unpacking stored cache slows down things a bit.
UV_HTTP_TIMEOUT is increased to 500 seconds, since sometimes uv might hang during package download. But when it works, it downloads them really fast.
As for right now it's not possible to do uv pip install ., so the workaround with installing packages in an editable mode (which is supported) is used here.

It's of course blazingly fast, but it doesn't support all the functionality of the pip.
For instance it cannot install requirements-all.txt:
(lit-gpt-uv) ➜  lit-gpt git:(autogptq_integration) uv pip install -r requirements-all.txt                                                                                INSERT
error: Unsupported requirement in requirements-all.txt at position 658
  Caused by: URL requirement must be preceded by a package name. Add the name of the package before the URL (e.g., `package_name @ https://...`).
git+https://github.com/EleutherAI/lm-evaluation-harness.git@115206dc89dad67b8beaa90051fb52db77f0a529
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I see that it's developing in a quite intensive manner, so I think we can revisit it in a month or so, but right now it's not a drop-in replacement for pip.

It doesn't know list command, only freeze.
(lit-gpt-uv) ➜  lit-gpt git:(autogptq_integration) uv pip list                                                                                INSERT
error: unrecognized subcommand 'list'

  tip: a similar subcommand exists: 'install'

Usage: uv pip [OPTIONS] <COMMAND>

For more information, try '--help'.

If to change requirements-all.txt

On my local machine, with all the packages being cached, uv reduced installation time from 67 seconds to 17.
On a CI install all dependencies step is reduced from 45 seconds to 28 seconds. Is it enough to justify the change?
But it throws some unexpected errors.
I'll revisit it when I have time. In the meantime, let's keep it as a Draft.

Not yet ready for integration, but close.
The biggest issues:

doesn't support uv pip install ., but you can install the package in edit mode by doing: uv pip install -e .. Not a hugely criminal workaround, yet not what I would like to do in a CI.
sometimes might hang during package download. But when it works, it does it really fast.

I'll try it one more time in a week.

Honestly I would be fine merging it now that it's working at least

By the way, we now support uv pip install . without -e, in the latest release.

Thanks for the heads up Charlie. I just gave it a try in #1187 but non-editable requires --prerelease=allow since we have a prerelease direct dependency.
But adding this makes dependencies of our dependencies also install prereleases. This in turn installs the current numpy prerelease (2.0.0b1) which itself doesn't work with current torch:
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0b1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled against NumPy 2.0.
If you are a user of the module, the easiest solution will be to
either downgrade NumPy or update the failing module (if available).
Traceback (most recent call last):  File "<string>", line 1, in <module>
  File "/home/runner/work/litgpt/litgpt/litgpt/__init__.py", line 6, in <module>
    from litgpt.model import GPT  # needs to be imported before config
  File "/home/runner/work/litgpt/litgpt/litgpt/model.py", line 12, in <module>
    import torch
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/torch/__init__.py", line 1477, in <module>
    from .functional import *  # noqa: F403
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/torch/functional.py", line 9, in <module>
    import torch.nn.functional as F
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/torch/nn/__init__.py", line 1, in <module>
    from .modules import *  # noqa: F403
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/torch/nn/modules/__init__.py", line 35, in <module>
    from .transformer import TransformerEncoder, TransformerDecoder, \
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/torch/nn/modules/transformer.py", line 20, in <module>
    device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: module compiled against ABI version 0x1000009 but this version of numpy is 0x2000000 (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)

Is there a way to set --prerelease= only for our direct dependencies? Maybe --prereleaes=allow-direct?

Ah yeah, we do allow prereleases for direct dependencies, but we treat . and -e . a little differently... For editables, we treat any dependency of an editable as "direct"; but for direct reference dependencies (like . or https://...), we consider that the direct dependency, and any of its dependencies are transitive. I think we can improve this: https://github.com/astral-sh/uv/issues/2643


Nested namespace accessor

I'm trying to remove stopwords from a list of strings with dask.bag.
This returns a list of the stopwords for each line:
b.pluck('abstract').map(lambda line: line.split()).map(lambda l: filter(STOPWORDS.__contains__, l)).take(1)

What's the best way to do the opposite (return if the word is not contained) with dask.bag?
I think I'm looking for something like:
https://docs.python.org/2/library/itertools.html#itertools.ifilterfalse
http://toolz.readthedocs.org/en/latest/api.html#toolz.itertoolz.remove
If an equivalent function is not implemented, what's the recommended alternative?
Thanks!

I solved it by using itertoolz the following way:
from toolz.itertoolz import remove
b.pluck('abstract').map(lambda line: line.split()).map(lambda l: list(remove(STOPWORDS.__contains__, l))).take(1)

Great.  By the way, you can replace map(lambda line: line.split()) with map(str.split).

Thanks @eriknw . I actually got this error when replacing (using 0.5.0) :
TypeError:  Exception in remote process

descriptor 'split' requires a 'str' object but received a 'unicode'

Traceback:
  File "/Users/cdoig/anaconda/envs/dask-tokenize/lib/python2.7/site-packages/dask/async.py", line 258, in execute_task
    result = _execute_task(task, data)
  File "/Users/cdoig/anaconda/envs/dask-tokenize/lib/python2.7/site-packages/dask/async.py", line 241, in _execute_task
    return func(*args2)

but I belive Matt fixed it here: https://github.com/mrocklin/dask/commit/0707698ffd26cc8b6656626f538a295d4fe0f21e

Good point, your way is more compatible.  Another way to be compatible is to import string ; ... ; map(string.split), but your way is good (and explicit) too.

There is a str namespace in bag so one can also do
b.str.split()

This works both on unicode and on strings.
I'll add remove in the morning (some 8 hours from now)

Yeah, @chdoig pointed to the commit that added str/unicode compatibility to the b.str accessor.  Her example above would be even nicer if there were also a list or generic sequence accessor, such as b.pluck('abstract').str.split().list.remove(STOPWORDS.__contains__).take(1).

I'm fairly comfortable dumping sequence functions into the dask.bag.Bag namespace.  Several other functions are already there, notably filter.

To clarify, I was thinking about an accessor, Bag.list, that behaves like Bag.str.  For example,
In [1]: import dask.bag as db

In [2]: iseven = lambda x: x % 2 == 0

In [3]: seq = [0, 1, 2, 3]

In [4]: list(db.from_sequence(seq).filter(iseven))
Out[4]: [0, 2]

In [5]: seq_of_seqs = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]

In [6]: list(db.from_sequence(seq_of_seqs).list.filter(iseven))
Out[6]: [[0, 2], [4], [6, 8]]

Oh!  Interesting.  One might presumably want to nest these.  That might be fun.
Looking again at the original error report I see that I misunderstood the problem.  Yes, this looks like a good idea.

Also short term, your question was how do we do this but with remove
b.pluck('abstract').map(lambda line: line.split()).map(lambda l: filter(STOPWORDS.__contains__, l)).take(1)

A trivial way would be to use remove
b.pluck('abstract').map(lambda line: line.split()).map(lambda l: remove(STOPWORDS.__contains__, l)).take(1)

Alternatively if you don't care about the list structure at this point then you could also concatenate your lists into one large bag
b.pluck('abstract').str.split().concat().remove(STOPWORDS.__contains__).take(1)

Matt,
Your suggestion:
b.pluck('abstract').map(lambda line: line.split()).map(lambda l: remove(STOPWORDS.__contains__, l)).take(1)

Gets me this error:
---------------------------------------------------------------------------
RemoteError                               Traceback (most recent call last)
<ipython-input-12-2025568b18c2> in <module>()
----> 1 b.pluck('abstract').str.split().map(lambda l: remove(STOPWORDS.__contains__, l)).take(1)

/Users/cdoig/dask/dask/bag/core.pyc in take(self, k, compute)
    565         b = Bag(merge(self.dask, dsk), name, 1)
    566         if compute:
--> 567             return tuple(b.compute())
    568         else:
    569             return b

/Users/cdoig/dask/dask/bag/core.pyc in compute(self, **kwargs)
    574     def compute(self, **kwargs):
    575         """ Force evaluation of bag """
--> 576         results = get(self.dask, self._keys(), **kwargs)
    577         if isinstance(results[0], Iterable):
    578             results = toolz.concat(results)

/Users/cdoig/dask/dask/bag/core.pyc in get(dsk, keys, get, **kwargs)
     86     dsk2 = optimize(dsk, keys)
     87
---> 88     return get(dsk2, keys, **kwargs)
     89 _get = get
     90

/Users/cdoig/dask/dask/multiprocessing.pyc in get(dsk, keys, optimizations, num_workers, func_loads, func_dumps)
     49         # Run
     50         result = get_async(apply_async, len(pool._pool), dsk2, keys,
---> 51                            queue=queue)
     52     finally:
     53         if cleanup:

/Users/cdoig/dask/dask/async.pyc in get_async(apply_async, num_workers, dsk, result, cache, debug_counts, queue, raise_on_exception, **kwargs)
    448         if isinstance(res, Exception):
    449             raise type(res)(" Exception in remote process\n\n"
--> 450                 + str(res) + "\n\nTraceback:\n" + tb)
    451         state['cache'][key] = res
    452         finish_task(dsk, key, state, results)

RemoteError:
---------------------------------------------------------------------------
 Exception in remote process

---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/cdoig/anaconda/envs/dask-text/lib/python2.7/multiprocessing/managers.py", line 240, in serve_client
    request = recv()
TypeError: ifilterfalse expected 2 arguments, got 0
---------------------------------------------------------------------------

Traceback:
  File "/Users/cdoig/dask/dask/async.py", line 270, in execute_task
    queue.put(result)
  File "<string>", line 2, in put
  File "/Users/cdoig/anaconda/envs/dask-text/lib/python2.7/multiprocessing/managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
---------------------------------------------------------------------------

It works if I add list() to remove, as the solution my second comment:
b.pluck('abstract').map(lambda line: line.split()).map(lambda l: list(remove(STOPWORDS.__contains__, l))).take(1)

@eriknw any further interest in this?
I'm now mildly in favor of just recommending that people continue to use lambdas.  This reduces maintenance burden and API on our end.

I don't plan to get to this in the near term.  Feel free to close the issue.  It is neat, though, and offers modest (but not amazing) performance boosts, so I may revisit or post a WIP branch if others are interested.

Closing for now as stale. Feel free to reopen if needed.


PyYAML dependency

It appears like this code requires installation of the PyYAML package, but this is not properly listed in requirements.
After installing deep pose kit following instructions, importing the package results in the error:
ModuleNotFoundError: No module named 'yaml'
manually doing
pip install pyyaml
fixes this

Thanks! Sorry I was traveling and just got back to work.


[CWS] upgrade opa rego to v0.37.1

What does this PR do?
This PR upgrades the rego evaluator dependency.
This bump was tested on the security-monitoring rego testsuite
Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] The appropriate team/.. label has been applied, if known.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the config template has been updated.

@mx-psi I asked a review from you because of the changes to other dependencies that this bump brings. One dependency in particular seems to be the prometheus client. Looking at the changelog it seems that v1.12.0 actually adds support for Go 1.17 so maybe we actually need it ?

I don't know which Prometheus features we use in the Agent, that is a question for Agent Core. If we use the feature update in prometheus/client_golang#955 I think we should update indeed. In any case, I think this PR leaves us better (or at least not worse) than we currently are, so I will approve it!


[ENHANCEMENT] Tasklist should use .Values.global.elasticsearch.url the same way Operate does

Describe the use case:
When customer wants to use SSL connection to Elasticsearch, Tasklist must be configured with the help of camunda.tasklist.elasticsearch.url and camunda.tasklist.zeebeElasticsearch.url, where the HTTPS scheme may be provided. Currently chart configuration is not picking the global .Values.global.elasticsearch.url into these params.
Describe the enhancement/feature:
It should work the same way as in Operate: https://github.com/camunda/camunda-platform-helm/blob/main/charts/camunda-platform/charts/operate/templates/configmap.yaml#L20-L21 and https://github.com/camunda/camunda-platform-helm/blob/main/charts/camunda-platform/charts/operate/templates/configmap.yaml#L38-L39

Related support issue: https://jira.camunda.com/browse/SUPPORT-14398


Set uid and gui on start_program in check

Hi,
I tried to set the uid and gid on the start_program property, but the resource ignored it. It be possible to run the start_program as a especified user?

Assuming you mean the monit_check resource, no it doesn't have specific support for this as it is very very simple. You would want to use the more generic monit_config resource which takes either a template or block of text directly so you can do anything Monit supports.

I see. That's probably a better way. Would be nice to be able to do it in the monit_check though. Thank for the head's up!

monit_check is specifically aimed at the use case of "I want to use Monit to watch a service from a distro package (or equivalent)". Since the commands for that will always be run as root it's probably outside of the scope. By the time you're making something that custom it's going to be clearer to just write it in-line I bet :)
Since I noticed I don't have an example of that in the readme:
``ruby
monit_config 'myservice' do
content <<-EOH
whatever config
you want can
go here
EOH
end


[ISSUE  #649]✅Fix DataVersion test case for data_version_next_version error

Which Issue(s) This PR Fixes(Closes)

Fixes #649
Brief Description

How Did You Test This Change?

🚀Thanks for your contribution🎉. CodeRabbit(AI) will review your code first🔥


Add clarification around how group and check configurations interact

Affected Components

[ ] Content & Marketing
[ ] Pricing
[ ] Test
[x] Docs
[ ] Learn
[ ] Other

@ragog Great thought, I made that change as well. Please re-review when you get a chance.

@cjanslow I highlighted the warnings on both pages in a bit of a different way. Later we might want to take another look at the Groups page in general as it is not CLI-first, but that's a different PR. Could you review?

@ragog These changes look great, makes the impact a lot more obvious! LGTM


Efficient Value Transfer - Payment Channels, Checkbooks, Probabilistic Micropayments

Livepeer's POC currently uses a deposit+claim payments system with custom logic for when broadcasters can withdraw their deposits and when transcoders get to draw from a broadcasters deposit. More efficient, secure, and attack-proof methods are likely available via generalized solutions like:

Payment channles, like Raiden
Checkbook contracts, like within Swarm
Probabilistic Micropayments, like in Orchid

These solutions should be considered in not only the context of transcoding, but also for fair compensation for bandwidth in P2P data transfer as referenced in #3

Related to this, but not exactly the same topic: efficient protocol-to-many fee release and payments. Each user needs to claim all the fees and rewards they're owed for each round, and this can get expensive. A mechanism like merkle-root based pool payments may be interesting to consider.
https://ethresear.ch/t/pooled-payments-scaling-solution-for-one-to-many-transactions/590
https://github.com/cardstack/merkle-tree-payment-pool
There are all sorts of challenges around making sure the proofs are available offchain, but also potential game-theoretic solutions involving staking, acceptance, and slashing in the case of invalid or non-available proofs. For example, transcoders (validators), could play an additional role in producing these proofs and accepting proposed ones, and then watchers could slash their deposits for any faults.


Extension pack dependencies are not respected when installing via the VS Code URI

Issue Type: Bug

Open a VSCode window
Open the uri vscode://shsuman.dummy-extension-main in the browser
VSCode will prompt you to install the Dummy Extension - Main
Install the Dummy Extension - Main (shsuman.dummy-extension-main)

This will install the Dummy Extension - Main but wouldn't install the Dummy Extension -Remote(shsuman.dummy-extension-remote) which is a part of the extension pack for the Dummy Extension - Main
VS Code version: Code 1.52.1 (ea3859d4ba2f3e577a159bc91e3074c5d85c0523, 2020-12-16T16:34:46.910Z)
OS version: Windows_NT x64 10.0.19042

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz (8 x 2112)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offopengl: enabled_onprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.92GB (5.42GB free)

Process Argv
--crash-reporter-id 611024cc-5acc-4121-a31a-6d32c5099117

Screen Reader
no

VM
0%

Extensions (21)

Extension
Author (truncated)
Version

gitlens
eam
11.1.3

tsl-problem-matcher
eam
0.3.1

EditorConfig
Edi
0.16.4

prettier-vscode
esb
5.8.0

vscode-azurestorage
ms-
0.11.0

vscode-docker
ms-
1.9.1

csharp
ms-
1.23.8

python
ms-
2020.12.424452561

vscode-pylance
ms-
2021.1.1

jupyter
ms-
2020.12.414227025

vscode-ai
ms-
0.6.20-dev

remote-ssh
ms-
0.62.0

remote-ssh-edit
ms-
0.62.0

remote-wsl
ms-
0.52.0

azure-account
ms-
0.9.4

vscode-typescript-tslint-plugin
ms-
1.3.3

vsliveshare
ms-
1.0.3484

java
red
0.73.0

html-preview-vscode
tht
0.2.5

vscode-java-debug
vsc
0.30.0

vscode-java-dependency
vsc
0.17.0

A/B Experiments
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyt875:30247033

I tried it and it install both extensions for me. Can you please try to repro again and let me know the exact steps you follow. It would be helpful if you provide a gif.

I tried it and it install both extensions for me. Can you please try to repro again and let me know the exact steps you follow. It would be helpful if you provide a gif.

Yes, I am seeing the right behaviour now. For some reason, it wasn't working at the time

Yes, I am seeing the right behaviour now. For some reason, it wasn't working at the time


ResourceDecorator.getResourceName() is not using the resource name

Describe the bug
Resources in the sidebar are always showing id and not name. The name() method is never being called on the resource extended from BaseResource.
Installed libraries and their versions
"admin-bro": "^2.4.2",
"admin-bro-expressjs": "^2.0.5",
"admin-bro-sequelizejs": "^0.4.4",

To Reproduce
Implement a custom resource that has different name and id values.
class Resource extends BaseResource {
	name() {
		return this.model.name;	// table name
	}

	id() {
		return `${this.model.migId}-${this.model.name}`;
	}

}

You would see only id() method being called, and never the name() method on the resource.
Expected behavior
name() method should be called on the resource and used in the sidebar.
Screenshots
Refer to the sidebar produced (the bug):

What it should look like (after the fixing the issue)

Fix / Solution
The problem is in the below code https://github.com/SoftwareBrothers/admin-bro/blob/6551b2b60c05f0f6c330bfd4a9d872573b2f9d13/src/backend/decorators/resource-decorator.ts#L157-L160
Note that in the current code both parameters are being sent as id(). The first parameter should be resource name, as shown below, which fixes the problem:
  getResourceName() {
    return this._admin.translateLabel(this._resource.name(), this.id());
  }

Everything can pretty much be translate in admin-bro. Including resource names. I guess that is why it is using the id rather than the name.
If you wish to change the name of the resource in the sidebar, set the name via the labels property on the AdminBroOptions

That does not fix the bug with the getResourceName method. It should be returning a name not id.  Check out the buggy code here - name is not being used at all.
https://github.com/SoftwareBrothers/admin-bro/blob/6551b2b60c05f0f6c330bfd4a9d872573b2f9d13/src/backend/decorators/resource-decorator.ts#L157-L160
This is the fix for the bug
  getResourceName() {
    return this._admin.translateLabel(this._resource.name(), this.id());
  }

For both sequelizejs and mongoose, id and name are essentially returning the same (tableName).
But I get now what you mean, in a custom adaptor where the name and id func is returning something different. I think your suggestion would work, but the other way around:
  getResourceName() {
    return this._admin.translateLabel(this.id(), this._resource.name());
  }

Since the id property can be customised, when using multiple resources with the same table name, for instance

originally BaseResource#name meant to return the human-readable name for a table. But along the way, we introduced i18n which moves all the human-readable things to AdminBroOptions.locale option. That is why BaseResource#name will be deprecated soon (as you pointed out it is not invoked anywhere).


[Question]: In a RAG pipeline, made using llamaindex, where is "llm" used while where is "embed model" used ?

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
    content = document["content"]
    doc = Document(text=content)
    node_parser = HierarchicalNodeParser.from_defaults(chunk_sizes=chunk_size)
    nodes = node_parser.get_nodes_from_documents([doc])
    service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model,node_parser=node_parser)
    storage_context = StorageContext.from_defaults()
    storage_context.docstore.add_documents(nodes)
    index = VectorStoreIndex(nodes, storage_context=storage_context, service_context=service_context)
    index.storage_context.persist(persist_dir=pathname)
    postproc = None
    reranker = SentenceTransformerRerank(top_n = rr_metadata)
    retriever = index.as_retriever(similarity_top_k=retrieval_metadata_similarity)
    retriever = AutoMergingRetriever(retriever,index.storage_context,verbose=True)
    response_mode = 'tree_summarize'
    response_synthesizer = get_response_synthesizer(response_mode=response_mode)
    node_postprocessors = [postproc, reranker]
    node_postprocessors = [processor for processor in node_postprocessors if processor is not None]
    query_engine = RetrieverQueryEngine(retriever, node_postprocessors=node_postprocessors)
    summary_whole = query_engine.query(rag_prompt_original)
    summary_text = str(summary_whole)  # Convert the summary object to a string
    wrapped_summary = textwrap.fill(summary_text, width=80)  # Wrap the summary text

Settings.llm = OpenAI(model="gpt-3.5-turbo", temperature=0.2)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small",)
If i give these two lines at the top of the code, will it ensure that embeddings are created using text embedding 3 small while the query engine uses llm ?
Any other things to take care at this stage, like in service context, emebdding model is also used separately?

Service context is deprecated. Settings controls the global defaults for things
You can also pass them into interfaces as needed instead of globals. For example, from your code
index = VectorStoreIndex(nodes=nodes, storage_context=storage_context, embed_model=embed_model)

# you didn't actually use this anywhere, but this is what uses the LLM in a query engine
response_synthesizer = get_response_synthesizer(response_mode=response_mode, llm=llm)

query_engine = RetrieverQueryEngine(retriever, node_postprocessors=node_postprocessors, response_synthesizer=response_synthesizer)

can i also pass llm to query_engine?
query_engine = RetrieverQueryEngine(retriever, node_postprocessors=node_postprocessors, response_synthesizer=response_synthesizer, llm=llm)
would it be redundant since llm is already passed in response synthesizer?

@GildeshAbhay exactly, the response synthesizer already has the LLM, so in the above the arg is unused


Error with novoalign

I got the following error when I tried running a variant calling pipeline with novoalign. I successfully ran the same sample with almost identical directory structure and yaml with bwa.
Terminal
[2017-02-16T17:09Z] Timing: alignment preparation
[2017-02-16T17:09Z] multiprocessing: prep_align_inputs
[2017-02-16T17:09Z] bgzip input file
[2017-02-16T17:10Z] bgzip input file
[2017-02-16T17:10Z] Resource requests: ; memory: 1.00; cores: 1
[2017-02-16T17:10Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job
[2017-02-16T17:10Z] Index input with grabix: 12_S12_L001_R1_001.fastq.gz
[2017-02-16T17:10Z] Index input with grabix: 12_S12_L001_R2_001.fastq.gz
[2017-02-16T17:11Z] multiprocessing: disambiguate_split
[2017-02-16T17:11Z] Timing: alignment
[2017-02-16T17:11Z] multiprocessing: process_alignment
[2017-02-16T17:11Z] Aligning lane NA12878-1 with novoalign aligner
Traceback (most recent call last):
  File "/usr/local/bin/bcbio_nextgen.py", line 4, in <module>
    __import__('pkg_resources').run_script('bcbio-nextgen==1.0.1', 'bcbio_nextgen.py')
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.6.7-py2.7.egg/pkg_resources/__init__.py", line 719, in run_script
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.6.7-py2.7.egg/pkg_resources/__init__.py", line 1483, in run_script
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-1.0.1-py2.7.egg-info/scripts/bcbio_nextgen.py", line 234, in <module>
    main(**kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-1.0.1-py2.7.egg-info/scripts/bcbio_nextgen.py", line 43, in main
    run_main(**kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 43, in run_main
    fc_dir, run_info_yaml)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 87, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 127, in variant2pipeline
    samples = run_parallel("process_alignment", samples)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"], batch_size=1)(joblib.delayed(fn)(x) for x in items):
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 804, in __call__
    while self.dispatch_one_batch(iterator):
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 662, in dispatch_one_batch
    self._dispatch(tasks)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 570, in _dispatch
    job = ImmediateComputeBatch(batch)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 183, in __init__
    self.results = batch()
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 51, in wrapper
    return apply(f, *args, **kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 84, in process_alignment
    return sample.process_alignment(*args)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/sample.py", line 116, in process_alignment
    data = align_to_sort_bam(fastq1, fastq2, aligner, data)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/alignment.py", line 64, in align_to_sort_bam
    names, align_dir, data)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/alignment.py", line 118, in _align_from_fastq
    out = align_fn(fastq1, fastq2, align_ref, names, align_dir, data)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/ngsalign/novoalign.py", line 61, in align_pipe
    out_file = os.path.join(align_dir, "{0}-sort.bam".format(dd.get_sample_name(data)))

YAML
  dir: ../final
details:
  - files: [../input/12_S12_L001_R1_001.fastq.gz, ../input/12_S12_L001_R2_001.fastq.gz]
    description: NA12878-1
    metadata:
      sex: female
    analysis: variant2
    genome_build: GRCh37
    algorithm:
      aligner: novoalign
      variantcaller: [freebayes, gatk-haplotype, samtools]
      mark_duplicates: false
      recalibrate: false
      realign: false
      coverage_interval: amplicon
      validate: giab-NA12878/truth_small_variants.vcf.gz
      validate_regions: giab-NA12878/truth_regions.bed
      variant_regions: ../input/v2-2_targets_sorted_no_chr.bed
      coverage: ../input/v2Exons_w_20bp_Padding.sorted.b37.bed
      svcaller: [cnvkit, seq2c]
      ensemble:
        numpass: 2

Thank you for the report and sorry about the issue. We were missing an import, triggering the error. I pushed a fix for this so if you update to the latest development version (bcbio_nextgen.py upgrade -u development) it should work cleanly. Thanks again.


Add games to favourites and switch between all games to favourites.

PR Description 📜
Add games to favourites and switch between all games to favourites.
Fixes #2923

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

Thank you @curator69 ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

hey @kunjgit whens the pr is getting merged??

Thank you  @curator69 , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀


Prepare for release 0.6.0: Updating version number

This change is

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Issues with Chroot in WSL

Latest, Creator's build

Using external software to mount ext4 drives, I attempt to chroot into my linux partion, Chroot fails with no error messages.
Expected: Chroot command completes and the external linux is loaded in root terminal.

Windows-exotic filesystems are not supported at present, but if you are able to get the btrfs driver for Windows set up (I don't believe that the driver has a signature compatible with secureboot =\ ), you should be able to get things working in that case, since btrfs presents itself as ntfs to the kernel, if I remember correctly.
Full support would require probably a Microsoft ext4 driver to be built for the NT Kernel, which is sort of a problem unrelated to WSL.

so it wouldn't be possible to not have a Windows driver but instead let Linux mount them??

Not right now.  Microsoft's support for LXFS (the rootfs and homefs for WSL) is a kind of virtual device driver built on top of a sort of internal VFS layer that emulates the full *nix semantics, and DrvFS is another virtual device driver that implements partial *nix semiantics for officially supported Windows filesystems (NTFS, Fat32, CDFS, Win32 network shares, etc).  True ext4 support needs both an underlying Windows driver as well as a special virtual driver for WSL to understand it.  This means you need both a filesystem driver for the NT Kernel as well as an additional driver to integrate that filesystem support into WSL.
If there were an official ext4 driver, the WSL team could probably add support on their own (in particular, they could probably implement full *nix semantics without emulation on top of it, since this is supported as part of the ext4 filesystem).  The problem is that they are unlikely to add support for a third-party filesystem driver.
Another option is to wait for them to open up their internal device driver API for lxcore, but this is still closed at the moment since it is still under heavy development.

@kenshen112 - can you post your repro steps. What is your external ext4 software, what command did you use to mount that drive, what was the result of mounting the drive, can you see your chroot directory hierarchy. After than point we can get to chroot failing with no errors, and possibly look at an strace.

I have written a script on my own personal github, and this issue is plaguing me. The steps as follow:
Mount the ext4 drive in a 3rd party program (The only one is Ext2Fsd.) in the drive "Z:"
Mount the ext4 partition in WSL mount -t drvfs Z: /mnt/z/
An error is thrown: "WARNING: drive write protected, mounted read-only."
This is an error in the host program Ext2Fsd not being able to mount the drive properly
Attempt the chroot chroot /mnt/z/ /bin/bash/
Nothing happens. No errors thrown but no output at all. Chroot immediately fails.
The issue is that the only 3rd party program to mount ext4 partitions cannot do it properly.

Here is the strace output of running chroot /mnt/z/ /bin/bash as listed above
https://pastebin.com/CTS6gTdd

I am further investigating the issue with the paid software Paragon - Linux Filesystems for Windows

I am further investigating the issue with the paid software Paragon - Linux Filesystems for Windows

Right, the other only 3rd party Ext2 NT Windows filesystem driver. Ext2Fsd is #579. There are still no repro steps given for your chroot scenario. The error you are seeing is:
execve("/bin/bash", ["/bin/bash"], [/* 26 vars */]) = -1 ENOEXEC (Exec format error)

I can't tell you whether that is a problem with your /bin/bash or whether it is to do with chroot into drvfs in general, because you didn't say how you populated /bin. This is somewhat academic however. Whether bash fires up initially or not, the next thing it would try to do is access /dev/tty, which Ext2Fsd and Paragon do not support. The very rough analogy here is you are trying to chroot into a ntfs-3g mount on Real Linux. Rather than jumping into chroot scenarios, start with basic file operations.
Duping into #579 to keep the "enter ext4" scenarios in one place. At this point whether using Ext2Fsd or Paragon you are looking at the same fundamental issues. [If at some point support was added for Ext2Fsd but not Paragon, or visa versa, we can start differentiating the finer points.]


Saving throw incorrectly calculated for one creature when importing from the GM notes

https://app.roll20.net/forum/permalink/4003477/
The Stone Dwarf
Medium humanoid (dwarf), neutral
Armor Class 15 (natural armor)
Hit Points 67 (9d8 + 27)
Speed 30 ft.
STR DEX CON INT WIS CHA
18 (+4) 16 (+3) 16 (+3) 10 (+0) 12 (+1) 15 (+2)
Saving Throws Str +6, Dex +5, advantage against poison
Skills Athletics +6, Intimidation +4
Senses passive Perception 13
Languages Common, Dwarfish
Challenge 3 (700 XP)
Brave. The stone dwarf has advantage on saving throws against being frightened.
Actions
Multiattack. The stone dwarf makes two hand axe attacks.
Hand Axe. Melee Weapon Attack: +6 to hit, reach 5 ft., one target. Hit: 7 (1d6 + 4) slashing damage.
this looks like it must be a regression in the sheetworker code that parses saving_throws_srd that was introduced somewhere between 6.0.4 and 6.3.0

Fixed in 6.3.1


dejitter key error

Hello,
I have noticed my app freezing occasionally on a raspberry pi 3 with capacitive touch screen (mtdev provider). When having dejitter enabled in the kivy config file, I sometimes get this error:
self.guiObject.run()
File "/usr/local/lib/python3.4/dist-packages/kivy/app.py", line 828, in run
runTouchApp()
File "/usr/local/lib/python3.4/dist-packages/kivy/base.py", line 487, in runTouchApp
EventLoop.window.mainloop()
File "/usr/local/lib/python3.4/dist-packages/kivy/core/window/window_egl_rpi.py", line 89, in mainloop
self._mainloop()
File "/usr/local/lib/python3.4/dist-packages/kivy/core/window/window_egl_rpi.py", line 84, in _mainloop
EventLoop.idle()
File "/usr/local/lib/python3.4/dist-packages/kivy/base.py", line 330, in idle
self.dispatch_input()
File "/usr/local/lib/python3.4/dist-packages/kivy/base.py", line 308, in dispatch_input
self.input_events = mod.process(events=self.input_events)
File "/usr/local/lib/python3.4/dist-packages/kivy/input/postproc/dejitter.py", line 61, in process
del self.last_touches[touch.id]
KeyError: 25

I suspect this is a bug. Does anyone have further information about this?
Maybe related to #3758
Using kivy 1.9.1.
Thank you.

fixed in https://github.com/kivy/kivy/pull/4646


Harvest restart changes instance causing some 7mode graphs to show extra entries.

Describe the bug
Affects 7-mode Dashboards only.
When harvest is restarted the ports for the pollers may change causing a differing instance.
The 7-mode queries as set, in aggregate, volume, and Network dashboards, will show multiple entries for the same object and create separate graph colors for the time span covered when a restart is within that time frame.
Minor annoyance.
Environment
CentOS7, Harvest latest nightly build, Grafana 8.2.1-1

Harvest version: harvest version 21.10.2000-nightly (commit 7334f11) (build date 2021-10-20T00:22:28-0400) linux/amd64
Command line arguments used: [e.g. bin/harvest start --config=foo.yml --collectors Zapi]
OS: CentOS 7
Install method: yum rpm
ONTAP Version: [7mode]
Other:

To Reproduce
Steps to reproduce the behavior:
Open any of the following Dashboards:
Aggregate 7 mode
Network 7 mode
volume 7 mode
Restart Harvest:
systemctl restart harvest
Expected behavior
Appearance in graphs shouldn't change with duplicate or more items.
Actual behavior
Items in graphs add extra entries for each restart within viewed time period.
Possible solution, workaround, fix
Add sum by(node,[appropreate_label])
Pull request on its way.
Additional context
Feel free to contact me directly by PM for direct access to my lab.
Legal Docs signed

Verified on f7be2f8
Setup six pollers to use port_range. Ran this script
for i in (seq 1 10);bin/harvest restart;sleep 30s;end

Each time Harvest was restarted, verified that all six pollers are assigned their respective port numbers. Validated there were no color changes in Disk dashboard.


Proposal for a contributing.md file

This readme includes these points :

Code of Conduct: Behavior guidelines.
Setup: Local installation steps.
Contributing: Bug reports, features, code.
Branches: Branching rules.
Style Guide: Code formatting rules.
Commits: Message format.
Testing: Running tests.
Pull Requests: Submission guidelines.
Reviews: PR review process.
License: Contributor legal info.

kindly assign me this issue

Hi @nishakp3005 Thank you for raising a detailed issue however this project is no longer in the gssoc or wob

nvm, I can still help you with it!

nvm, I can still help you with it!

okh I appreciate it.


Added missing migration

@claudio-rodriguez-dev you missed to upload the migration and I missed that you missed


Project templates not refreshing when new template added

The project templates view under organization settings shows only the newly added template after creation. Expected behaviour; all project templates of the current organization are listed.

Readme

Bug


Allow unused keys starting with cargo-

This change would make it so that cargo does not warn about "unused
keys" for keys that start with cargo-. This could be useful for custom
subcommands that want to be able to read settings from Cargo.toml.

r? @wycats
(rust_highfive has picked a reviewer for you, use r? to override)

#2668 implements a similar approach

Thanks for the PR! Talking about this in the past, however, we've preferred the approach in https://github.com/rust-lang/cargo/pull/2668 to this. Would that suffice for your use case?


Bump golang-1-linux to version v0.131.0

This is an automatically generated Pull Request from the Cryogenics CI Bot.
I have detected a new version of golang-release and automatically bumped
this package to benefit from the latest changes.
If this does not look right, please reach out to the #mapbu-cryogenics team.

Closing due to conflicts. CI should re-create.


[rfe_id:3423][crit:high][vendor:cnv-qe@redhat.com][level:component]VmWatch [test_id:3466]Should update vmi status with the proper columns using 'kubectl get vmi -w'

/triage build-watcher
/kind bug
What happened:
[rfe_id:3423][crit:high][vendor:cnv-qe@redhat.com][level:component]VmWatch [test_id:3466]Should update vmi status with the proper columns using 'kubectl get vmi -w'
logs here
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

KubeVirt version (use virtctl version):
Kubernetes version (use kubectl version):
VM or VMI specifications:
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

/reopen
Happened again https://prow.apps.ovirt.org/view/gcs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-latest/1363609743750336512


memcached: 1.5.11 -> 1.5.12

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/memcached/versions.
meta.description for memcached is: '"A distributed memory object caching system"'.

Checks done (click to expand)

built on NixOS
/nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12/bin/memcached passed the binary check.
1 of 1 passed binary check by having a zero exit code.
1 of 1 passed binary check by having the new version present in output.
found 1.5.12 with grep in /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12
directory tree listing: https://gist.github.com/a4e541f8f46eb9ce1b951c6ca83926ed
du listing: https://gist.github.com/20b30c98864688cf7771a040af9c7a40

Outpath report (click to expand)

Outpath difference report
4 total rebuild paths
1 package rebuilds
1 x86_64-linux rebuilds
1 i686-linux rebuilds
1 x86_64-darwin rebuilds
1 aarch64-linux rebuilds
First ten rebuilds by attrpath
memcached

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A memcached https://github.com/r-ryantm/nixpkgs/archive/1f6d946e656fdc1fe83df4113b95a7ed8e6cb0ff.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12
ls -la /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12/bin

cc @coreyoconnor for testing.

Success on x86_64-darwin (full log)
Attempted: memcached
Partial log (click to expand)
 /nix/store/acnfbicd84bnya2l2dq868b5l482qihw-coreutils-8.30/bin/install -c -m 644 protocol_binary.h '/nix/store/r7sa1ljkd97nv98wgkhvgr8klj0waaxl-memcached-1.5.12/include/memcached'
make[3]: Leaving directory '/private/tmp/nix-build-memcached-1.5.12.drv-0/memcached-1.5.12'
make[2]: Leaving directory '/private/tmp/nix-build-memcached-1.5.12.drv-0/memcached-1.5.12'
make[1]: Leaving directory '/private/tmp/nix-build-memcached-1.5.12.drv-0/memcached-1.5.12'
post-installation fixup
gzipping man pages under /nix/store/r7sa1ljkd97nv98wgkhvgr8klj0waaxl-memcached-1.5.12/share/man/
strip is /nix/store/g5r4apl0za012ffs6ladinwa5w0m1l3k-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/r7sa1ljkd97nv98wgkhvgr8klj0waaxl-memcached-1.5.12/bin
patching script interpreter paths in /nix/store/r7sa1ljkd97nv98wgkhvgr8klj0waaxl-memcached-1.5.12
/nix/store/r7sa1ljkd97nv98wgkhvgr8klj0waaxl-memcached-1.5.12

Success on aarch64-linux (full log)
Attempted: memcached
Partial log (click to expand)
make[1]: Leaving directory '/build/memcached-1.5.12'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12
shrinking /nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12/bin/memcached
gzipping man pages under /nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12/share/man/
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12/bin
patching script interpreter paths in /nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12
checking for references to /build in /nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12...
/nix/store/h9k2xx3kz8075irgjapd344p9h9r9j2y-memcached-1.5.12

Success on x86_64-linux (full log)
Attempted: memcached
Partial log (click to expand)
make[1]: Leaving directory '/build/memcached-1.5.12'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12
shrinking /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12/bin/memcached
gzipping man pages under /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12/share/man/
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12/bin
patching script interpreter paths in /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12
checking for references to /build in /nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12...
/nix/store/i1pg0y0j1zvhwbs1h3gj0sis4djdcicj-memcached-1.5.12


Adds BUPGEN image names stripping from TEGRA_BUPGEN_STRIP_IMG_NAMES variable

Adds a built-in way to strip image names from partition layout while generating the BUP payload.

OK, I'll implement the requested change

OK, I'll implement the requested change

I tweaked these changes a bit further and merged them manually into dunfell-l4t-r32.4.3, gatesgarth, and master.
Thanks @lfdmn !

I tweaked these changes a bit further and merged them manually into dunfell-l4t-r32.4.3, gatesgarth, and master.
Thanks @lfdmn !


Cannot update scratch.rb

General troubleshooting steps

[x] I have retried my command with --force and the issue is still present.
[x] I have checked the instructions for reporting bugs.

[x] None of the templates was appropriate for my issue, or I’m not sure.

[x] I ran brew update-reset && brew update and retried my command.
[x] I ran brew doctor, fixed as many issues as possible and retried my command.
[x] I understand that if I ignore these instructions, my issue may be closed without review.

Description of issue
Cannot run upgrade properly because of scratch.

Command that failed
brew cask upgrade

Output of command with --force --verbose --debug
$ brew cask upgrade scratch --verbose --debug
==> Upgrading 1 outdated package, with result:
scratch 2.0,460
==> Started upgrade process for Cask scratch
==> Hbc::Installer#fetch
==> Satisfying dependencies
All Cask dependencies satisfied.
==> Downloading
==> Downloading https://scratch.mit.edu/scratchr2/static/sa/Scratch-460.dmg
Already downloaded: /Users/admin/Library/Caches/Homebrew/Cask/scratch--2.0,460.dmg
==> Downloaded to -> /Users/admin/Library/Caches/Homebrew/Cask/scratch--2.0,460.dmg
==> Verifying download
==> Determining which verifications to run for Cask scratch
==> Checking for verification class Hbc::Verify::Checksum
==> 1 verifications defined
Hbc::Verify::Checksum
==> Running verification of class Hbc::Verify::Checksum
==> Verifying checksum for Cask scratch
==> SHA256 checksums match
==> Starting upgrade for Cask scratch
==> Un-installing artifacts
==> 2 artifact/s defined
#<SortedSet:0x000001030bbcc0>
==> Un-installing artifact of class Hbc::Artifact::Uninstall
==> Running uninstall process for scratch; your password may be necessary
==> Running uninstall script /Applications/Utilities/Adobe AIR Application Installer.app/Contents/MacOS/Adobe AIR Application Installer
==> Executing: ["/bin/chmod", "--", "+x", "/Applications/Utilities/Adobe AIR Application Installer.app/Contents/MacOS/Adobe AIR Application Installer"]
==> Executing: ["/usr/bin/sudo", "-E", "--", "/Applications/Utilities/Adobe AIR Application Installer.app/Contents/MacOS/Adobe AIR Application Installer", "-uninstall", "-silent", "/Applications/Scratch 2.app"]
==> unexpected error: Error: Missing necessary file META-INF/AIR/application.xml
==> Purging files for version 2.0,460 of Cask scratch
Error: Command failed to execute!

==> Failed command:
/usr/bin/sudo -E -- /Applications/Utilities/Adobe AIR Application Installer.app/Contents/MacOS/Adobe AIR Application Installer -uninstall -silent /Applications/Scratch 2.app

==> Standard Output of failed command:
unexpected error: Error: Missing necessary file META-INF/AIR/application.xml

==> Standard Error of failed command:

==> Exit status of failed command:
#<Process::Status: pid 84012 exit 5>

/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:71:in `assert_success'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:36:in `run!'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:14:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_uninstall.rb:204:in `uninstall_script'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_uninstall.rb:58:in `block in dispatch_uninstall_directives'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_uninstall.rb:55:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_uninstall.rb:55:in `dispatch_uninstall_directives'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/uninstall.rb:7:in `uninstall_phase'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:403:in `block in uninstall_artifacts'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/set.rb:674:in `each'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/set.rb:674:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:400:in `uninstall_artifacts'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:362:in `start_upgrade'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/upgrade.rb:59:in `block in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/upgrade.rb:29:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/upgrade.rb:29:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:35:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:98:in `run_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:168:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:132:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:101:in `<main>'
Error: Kernel.exit
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:173:in `exit'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:173:in `rescue in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:156:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:132:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:101:in `<main>'

Output of brew cask doctor
$ brew cask doctor
==> Homebrew-Cask Version
Homebrew-Cask 1.6.3-11-g9c3043c
caskroom/homebrew-cask (git revision 54bf7; last commit 2018-05-08)
==> macOS
10.13.4
==> SIP
Enabled
==> Java
1.8.0_144
==> Homebrew-Cask Install Location
<NONE>
==> Homebrew-Cask Staging Location
/usr/local/Caskroom
==> Homebrew-Cask Cached Downloads
~/Library/Caches/Homebrew/Cask (4 files, 115MB)
==> Homebrew-Cask Taps:
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask (3961 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-drivers (145 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-fonts (1161 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-versions (176 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-eid (11 casks)
==> Contents of $LOAD_PATH
/usr/local/Homebrew/Library/Homebrew/cask/lib
/usr/local/Homebrew/Library/Homebrew
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/gems/2.3.0/gems/did_you_mean-1.0.0/lib
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/site_ruby/2.3.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/site_ruby/2.3.0/x86_64-darwin9.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/site_ruby/2.3.0/universal-darwin9.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/site_ruby
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/vendor_ruby/2.3.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/vendor_ruby/2.3.0/x86_64-darwin9.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/vendor_ruby/2.3.0/universal-darwin9.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/vendor_ruby
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/x86_64-darwin9.0
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/universal-darwin9.0
==> Environment Variables
LC_ALL="en_US.UTF-8"
PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/Homebrew/Library/Homebrew/shims/scm"
SHELL="/bin/bash"

Output of brew tap
$ brew tap
buo/cask-upgrade
busterc/tap
caskroom/cask
caskroom/drivers
caskroom/eid
caskroom/fonts
caskroom/versions
domt4/autoupdate
homebrew/bundle
homebrew/command-not-found
homebrew/core
homebrew/services
kitsuyui/myip
timsutton/formulae
vitorgalvao/tiny-scripts

brew cask uninstall --force scratch || rm -rf "$(brew --prefix)/Caskroom/scratch"
brew install --force scratch

Still the same error:
brew install --force scratch
==> Uninstalling Cask scratch
==> Running uninstall process for scratch; your password may be necessary
==> Running uninstall script /Applications/Utilities/Adobe AIR Application Installer.app/Contents/MacOS/Adobe AIR Application Installer
==> unexpected error: Error: Missing necessary file META-INF/AIR/applicati
Error: Command failed to execute!

==> Failed command:
/usr/bin/sudo -E -- /Applications/Utilities/Adobe AIR Application Installer.app/Contents/MacOS/Adobe AIR Application Installer -uninstall -silent /Applications/Scratch 2.app

==> Standard Output of failed command:
unexpected error: Error: Missing necessary file META-INF/AIR/application.xml

==> Standard Error of failed command:

==> Exit status of failed command:
#<Process::Status: pid 52657 exit 5>
$ brew install --force scratch
Error: Another active Homebrew update process is already in progress.
Please wait for it to finish or terminate it to continue.
Error: No available formula with the name "scratch"
==> Searching for a previously deleted formula (in the last month)...
Error: No previously deleted formula found.
==> Searching for similarly named formulae...
==> Searching local taps...
Error: No similarly named formulae found.
==> Searching taps...
==> Searching taps on GitHub...
This formula was found in a tap:
caskroom/cask/scratch
To install it, run:
  brew install caskroom/cask/scratch

After running brew install caskroom/cask/scratch the issue was solved.


How to load polymorphic class without knowing its polymorphic_id?

Dear developers,
I would like to load an XML archive into a polymorphic pointer, therefore I followed the examples on the page https://uscilab.github.io/cereal/polymorphism.html .
But, I would like to be able to load a fill without knowing the polymorphic_id value which is internal to cereal, since in my application the input XML files are filled by users not by computers.
So I could reinject the polymorphic_id information by a postprocessing of my input XML file.
This means that I need to have access to the map linking the polymorphic_id to the classes names.
How can I have access to this map ? Or is it possible to load a structure into a polymorphic pointer without fillling the polymorphic_id ?
Thanks by advance.
PS: these questions are strongly related to https://stackoverflow.com/questions/68655182/c-cereal-how-to-load-polymorphic-class-without-knowing-its-polymorphic-id

Sorry it was a misunderstanding of how cereal works.


Merkle Mountain Range (MMR)

Summary
MMR  is a data structure designed to address limitations related to Merkle Trees. It is essential in scenarios where on-chain data updates are required. MMRs are commonly used in technologies like zk-coprocessors and are particularly valuable in non-interactive state proofs. While MMRs are more gas-expensive compared to Merkle Trees, in Mina there is a fixed-fee gas model that depends on network demand rather than the amount of computation.
Submitted by d0x471b
Ready to build?
Please read the docs on how to progress.

@hgedia Hi there, we are preparing a proposal for this, will send in a few weeks.

@codekaya Has started work on this primitive here.


Reload CDN client

Description
reload cdn client from dynamic config

Related Issue

Motivation and Context

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation Update (if none of the other choices apply)

Checklist:

[ ] Code compiles correctly.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Codecov Report

Merging #566 (89eaa49) into main (f73fc50) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #566      +/-   ##
==========================================
- Coverage   39.27%   39.27%   -0.01%
==========================================
  Files          86       86
  Lines        6640     6641       +1
==========================================
  Hits         2608     2608
- Misses       3662     3663       +1
  Partials      370      370

Flag
Coverage Δ

e2etests
?

unittests
39.27% <0.00%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

scheduler/config/config.go
4.44% <0.00%> (-0.05%)
:arrow_down:

scheduler/config/dynconfig.go
56.79% <ø> (ø)


Pin the Python SDK >= 16.4.1 for OpenSSL fix

KSM-333
Python SDK includes update of cryptography module which ...
Updated Windows, macOS, and Linux wheels to be compiled with
OpenSSL 3.0.7, which resolves CVE-2022-3602 and CVE-2022-3786.

This pins the SDK version in order to get those fixes. OpenSSL 3.0.7 still needs to be updated on the hosting operating system.

GHA did not trigger :(

No, the GHA tests. I made a PR and the tests didn't start. I closed the PR and created another and they triggered.


new(ci): added ability to run the update-kernels gha for a single distro to choose from.

What type of PR is this?
/kind feature

Any specific area of the project related to this PR?
/area ci

What this PR does / why we need it:
This PR introduces the possibility to run the update-kernels gha for a specific distro that can be selected when dispatching the job. This is useful to save time when we only want to crawl for a specific distro.
Which issue(s) this PR fixes:

NONE
Special notes for your reviewer:
NONE

Thanks for this!!
/hold

LGTM label has been added.  Git tree hash: 342dfa4e077f9b37b7c36d8262bd8eed22de44c5

/unhold


Article - keyline

Top align key line which divides columns with the top of key line under article author

Fixed with #298
See:  https://cardigan.wellcomecollection.org./components/preview/article-template.html


docker cp on OS X 10.11 fails to copy contents of directories recursively

i get a "permission denied" error when copying contents of subdirectories...
mmatiaschek@Markus-MBP ~/Work> docker exec d8 ls -al /var/www/html/sites/default
total 76
dr-xr-xr-x  3 www-data www-data  4096 Oct 25 10:04 .
drwxr-xr-x  5 www-data www-data  4096 Oct 25 10:11 ..
-rw-r--r--  1 www-data www-data  5997 Oct 21 22:51 default.services.yml
-rw-r--r--  1 www-data www-data 28086 Oct 21 22:51 default.settings.php
drwxrwxr-x 10 www-data www-data  4096 Oct 25 10:09 files
-r--r--r--  1 www-data www-data 28609 Oct 25 10:06 settings.php
mmatiaschek@Markus-MBP ~/Work> rm -rf 8/*
mmatiaschek@Markus-MBP ~/Work> docker cp d8:/var/www/html/sites/ 8/
open /Users/mmatiaschek/Work/8/sites/default/default.services.yml: permission denied
mmatiaschek@Markus-MBP ~/Work> ls -al 8/sites/default/
total 0
dr-xr-xr-x  2 mmatiaschek  staff   68 25 Okt 11:04 .
drwxr-xr-x  4 mmatiaschek  staff  136 25 Okt 12:05 ..
mmatiaschek@Markus-MBP ~/Work> ls -al 8/sites/
total 8
drwxr-xr-x  4 mmatiaschek  staff  136 25 Okt 12:05 .
drwxr-xr-x  3 mmatiaschek  staff  102 25 Okt 12:05 ..
-rw-r--r--  1 mmatiaschek  staff  515 22 Okt 00:51 README.txt
dr-xr-xr-x  2 mmatiaschek  staff   68 25 Okt 11:04 default
mmatiaschek@Markus-MBP ~/Work> docker --version
Docker version 1.8.2, build 0a8c2e3

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

I'm closing this as it's impossible to get anything without particular versions.


Implement changes for Symfony 2.7+3.0

Symfony added deprecated warnings, which cause our tests to fail.
We can try to just ignore them, by adjusting error_reporting, but as we want to keep supporting 2.3 and later on 3.0, it might be better to fix it now.
I haven't looked in details to all warnings, but it mostly seems related to service definitions using factory-method.
An idea we had, is to split some of the service definitions into separate files, and load the appropriate file from the BroadwayExtension.
We can use imports to share definitions that aren't affected by the deprecation warnings.

https://github.com/qandidate-labs/broadway/pull/167

For the factory/factory-[service|method] issue could you not do like a lot of others have done and moved that part into the Extension for the time being? Then when support is dropped for 2.3 move the definitions back into the individual config files.
For example...
$definition = $container->getDefinition('broadway.saga.state.mongodb_database');

// Required until support for 2.3 dropped
if (method_exists($def, 'setFactory')) {
    $definition->setFactory(array(
        new Reference('broadway.saga.state.mongodb_connection'),
        'selectDatabase'
    ));
} else {
    $definition->setFactoryService('broadway.saga.state.mongodb_connection');
    $definition->setFactoryMethod('selectDatabase');
}

Just ran into this when trying to use Broadway in Symfony 3:

  [Symfony\Component\DependencyInjection\Exception\InvalidArgumentException]
  Unable to parse file "/vendor/broadway/broadway/src/Broadway/Bundle/BroadwayBundle/DependencyInjection/../Resources/config/saga/mongodb.xml".

  [InvalidArgumentException]
  [ERROR 1866] Element '{http://symfony.com/schema/dic/services}service', attribute 'f
  actory-service': The attribute 'factory-service' is not allowed.
  [ERROR 1866] Element '{http://symfony.com/schema/dic/services}service', attribute 'f
  actory-method': The attribute 'factory-method' is not allowed.
  [ERROR 1866] Element '{http://symfony.com/schema/dic/services}service', attribute 'f
  actory-service': The attribute 'factory-service' is not allowed.
  [ERROR 1866] Element '{http://symfony.com/schema/dic/services}service', attribute 'f
  actory-method': The attribute 'factory-method' is not allowed.

The factory-service/factory-method setup was changed to the factory tag mid December with this commit but it only seems to be available in dev-master at this point so I would recommend using that.

Thanks for the tip!

solved by https://github.com/qandidate-labs/broadway/pull/228


Crash when combining type annotations for delayed functions with multiprocessing

Example problematic code snippet:
import typing

import dask

@dask.delayed
def blah() -> typing.List[int]:
    return [5]

blah().compute(scheduler='processes')

This results in a crash; traceback below. Instead running with scheduler='threads' works fine. Removing the -> typing.List[int] works fine with scheduler='processes'. Works fine regardless on Mac (Python 3.6.4 and 3.6.6). Weird!
Crash traceback (Python 3.6.5, Ubuntu 18.04):
TypeError                                 Traceback (most recent call last)
<ipython-input-1-fc913fc1b7b2> in <module>()
      7     return [7]
      8
----> 9 blah().compute(scheduler='processes')

~/.local/lib/python3.6/site-packages/dask/base.py in compute(self, **kwargs)
    154         dask.base.compute
    155         """
--> 156         (result,) = compute(self, traverse=False, **kwargs)
    157         return result
    158

~/.local/lib/python3.6/site-packages/dask/base.py in compute(*args, **kwargs)
    393     keys = [x.__dask_keys__() for x in collections]
    394     postcomputes = [x.__dask_postcompute__() for x in collections]
--> 395     results = schedule(dsk, keys, **kwargs)
    396     return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)])
    397

~/.local/lib/python3.6/site-packages/dask/multiprocessing.py in get(dsk, keys, num_workers, func_loads, func_dumps, optimize_graph, **kwargs)
    170                            get_id=_process_get_id, dumps=dumps, loads=loads,
    171                            pack_exception=pack_exception,
--> 172                            raise_exception=reraise, **kwargs)
    173     finally:
    174         if cleanup:

~/.local/lib/python3.6/site-packages/dask/local.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, **kwargs)
    499                         _execute_task(task, data)  # Re-execute locally
    500                     else:
--> 501                         raise_exception(exc, tb)
    502                 res, worker_id = loads(res_info)
    503                 state['cache'][key] = res

~/.local/lib/python3.6/site-packages/dask/compatibility.py in reraise(exc, tb)
    109     def reraise(exc, tb=None):
    110         if exc.__traceback__ is not tb:
--> 111             raise exc.with_traceback(tb)
    112         raise exc
    113

~/.local/lib/python3.6/site-packages/dask/local.py in execute_task()
    269     """
    270     try:
--> 271         task, data = loads(task_info)
    272         result = _execute_task(task, data)
    273         id = get_id()

/usr/lib/python3.6/typing.py in __new__()
    946             for base in bases:
    947                 if base is Generic:
--> 948                     raise TypeError("Cannot inherit from plain Generic")
    949                 if (isinstance(base, GenericMeta) and
    950                         base.__origin__ is Generic):

TypeError: Cannot inherit from plain Generic

In the same scenario, I also sometimes (not sure under what conditions...) get the following traceback instead:
  File "/tmp/tmpnlkkwnt3_pyzip/dask/base.py", line 156, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/tmp/tmpnlkkwnt3_pyzip/dask/base.py", line 395, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/tmp/tmpnlkkwnt3_pyzip/dask/multiprocessing.py", line 172, in get
    raise_exception=reraise, **kwargs)
  File "/tmp/tmpnlkkwnt3_pyzip/dask/local.py", line 501, in get_async
    raise_exception(exc, tb)
  File "/tmp/tmpnlkkwnt3_pyzip/dask/compatibility.py", line 111, in reraise
    raise exc.with_traceback(tb)
  File "/tmp/tmpnlkkwnt3_pyzip/dask/local.py", line 271, in execute_task
    task, data = loads(task_info)
AttributeError: type object 'weakref' has no attribute '__callback__'

I can reproduce this with only cloudpickle as follows:
import typing
def blah() -> typing.List[int]:
    return [5]
import cloudpickle
cloudpickle.loads(cloudpickle.dumps(blah))

I encourage you to raise this upstream at https://github.com/cloudpipe/cloudpickle/issues/new

Ah, I tried reproducing it with normal pickle and didn't see an error. Was unaware of cloudpickle. Will do, thanks.

Thanks for the minimal reproducible example by the way.  I'm going to close this for now.  Please speak up if something else comes up.

https://github.com/cloudpipe/cloudpickle/issues/200


4GB it's max size index table? Really?

My first using Sphinx and Thinking Sphinx... Ohh... After run my first rake task ts:rebuild I'm see this message:

indexing index 'xdocument_core'...
ERROR: index 'xdocument_core': too many string attributes (current index format allows up to 4 GB).
total 1425236 docs,<PHONE_NUMBER> bytes
total 331.702 sec, 10644895 bytes/sec, 4296.72 docs/sec

Many string attributes.... Ok, I'm read manual and added crc32, but it's not solved this problem. This my indices:
ThinkingSphinx::Index.define :xdocument, delta: true, with: :active_record do
  indexes url,        sortable: true
  indexes content,    sortable: true
  indexes sysname,    sortable: true
  indexes is_checked,      sortable: true
  indexes source_content,  sortable: true

  indexes author.name,        as: :field_author_name, sortable: true
  indexes author.ip,          as: :field_author_ip
  indexes author.description, as: :field_author_comment

  has id,         as: :attr_sort_document_id,         type: :integer
  has created_at, as: :attr_sort_document_created_at, type: :timestamp
  has empl_name,  as: :attr_sort_document_empl_name,  type: :string
  has sys_role,   as: :attr_sort_document_sys_role,   type: :string
  has sys_type,   as: :attr_sort_document_sys_type,   type: :string

  has author.room_num,                   as: :attr_sort_author_room_num,         type: :integer
  has 'CRC32(authors.team)',             as: :attr_sort_author_team,             type: :string
  has 'CRC32(authors.country)',          as: :attr_sort_author_country,          type: :string
  has 'CRC32(authors.city)',             as: :attr_sort_author_city,             type: :string
  has 'CRC32(authors.group_name)',       as: :attr_sort_author_group_name,       type: :string
  has 'CRC32(authors.type_machi)',       as: :attr_sort_author_type_machi,       type: :string
  has 'CRC32(authors.office_type)',      as: :attr_sort_author_office_type,      type: :string
  has 'CRC32(authors.version_product)',  as: :attr_sort_author_version_product,  type: :string
  has author.last_change_at,             as: :attr_sort_author_last_change_at, type: :timestamp
end

and this model:
class Xdocument < ApplicationRecord
  belongs_to :author

  # .... scopes
end

class Author < ApplicationRecord
  has_many :x_documents, dependent: :delete_all

  # .... scopes
end

I'm trying make a big filter like this one for example:
https://www.drupal.org/files/project-images/exposed_form_layout_0.png

Maybe there are some tips? I have two tables with data. In the first about 300 million documents (Xdocuments table), and in the second about 20 thousand items (authors table).
I hope for your help, because I can not solve the problem for many hours ...
P.S. debian 8.1, sphinx 2.3.2-id64-beta, thinking sphinx - last release.

I didn't realise Sphinx had a limit, but still, I think there's potential to change your index to reduce the size.
For every field where you're specifying :sortable => true, it's worth considering whether you actually need to sort by that field. When they're sortable, there's also a corresponding string attribute. For example: are you sorting by content anywhere? source_content? They sound like large text fields, and I find those are rarely useful for sorting.
Also, for your CRC'd columns, there's two things to note. Firstly, the CRC32 function will return an integer, so you can set the attribute types to integer rather than string. Secondly, if you want to filter by those specific values, the versions of TS and Sphinx that you're using should allow that to happen - and so, you don't actually need to use the CRC32 approach.
One more thing, though this is unrelated to string attributes: is is_checked a boolean column? If so, I think it'd make more sense as an attribute, rather than a field. Generally it only makes sense to use string/text columns for fields, as they're the textual data that Sphinx matches query terms on. Anything that's only used for sorting/filtering (the common use-case for boolean values) should be an attribute.

Can there be other solutions? Split indexes or change the limit? Now this is an incomplete table. It contains about 5-10% of all data (it is planned that it will be about 500-900GB of text for one table).

For every field where you're specifying :sortable => true, it's worth considering whether you actually need to sort by that field. When they're sortable, there's also a corresponding string attribute. For example: are you sorting by content anywhere? source_content? They sound like large text fields, and I find those are rarely useful for sorting.

Yes, many fields for sorting (from 10 and more), and many fields for search (6 for search and about 12 for filtering). Text fields can sometimes contain as one word, so can contain 10 megabytes of text. Yes, I understand that it is idiocy, but alas - such a legacy.

Also, for your CRC'd columns, there's two things to note. Firstly, the CRC32 function will return an integer, so you can set the attribute types to integer rather than string. Secondly, if you want to filter by those specific values, the versions of TS and Sphinx that you're using should allow that to happen - and so, you don't actually need to use the CRC32 approach.

It did not help :(

Text fields can sometimes contain as one word, so can contain 10 megabytes of text. Yes, I understand that it is idiocy, but alas - such a legacy.

The joys of legacy systems. One workaround could be to create truncated versions of these long columns? How accurate does the sorting need to be - down to 10 characters? 20? Could help reduce the index size.

Split indexes or change the limit?

I don't think this limit is configurable, but yes, you could potentially have many indices. You'd need to do that in a somewhat manual sense:
# tweak these numbers, or possibly source them
# from the db based on the total number of records?
shard_size = 1_000_000
shards = 300

shards.times do |shard|
  ThinkingSphinx::Index.define :xdocument, delta: true, name: "xdocument_{shard}", with: :active_record do
    # ...

    where "id > #{shard * shard_size} AND id <= #{(shard + 1) * shard_size}"
  end
end

I have a feeling any update to a single record with deltas will result in delta updates to all shard indices, though. Still, maybe it's worth it, if you can find the right balance of the number of shards?

I solved the problem... Removed sorting from content field and sysname. Now all is well. The customer still wants sorting, but I think I will be better to sort the search result. Then I will need to sort only 10 000 items. But I guess I still do shards, as you suggested. Just in case.
@pat , thanks for the help. Tell me, where you can see the list of rake tasks? I created the configuration (ts:rebuild) and did the indexing (ts:index). Now I want to start a reindexing delta indexes, but ts:in:delta not working. Then I tried the ts:index, but started a full indexing of the data.
Also, I do not understand where a rake command: merge and rotate.

Great to hear you've found an approach that works. I've answered the delta questions in #1080, so I'll close this issue.

Thanks for the help!