e2e unify the test framework

There are currently two variations, one based on bash, one based on go.
Any change needs to go in two places.
1.We need to switch to a single framework, preferably the one in go.

Decouple the tests from the istio templates. Tests need to test the istio.yaml file only, or the result of the script that generates the istio.yaml, they should not process templates individually, nor generate anything. Currently any new template needs to have a corresponding change in the e2e.

nope ... closing.


Relay Example error

when i run relay example in https://github.com/libp2p/go-libp2p-examples/tree/master/relay i get two errors

	h1, err := libp2p.New(context.Background(), libp2p.EnableRelay(circuit.OptDiscovery))  error is
circuit.OptDiscovery is now a no-op: dialing peers with a random relay is no longer supported
	relayaddr, err := ma.NewMultiaddr("/p2p-circuit/p2p/"+ h3.ID().Pretty()) report an't dial a p2p-circuit without specifying a relay: /p2p-circuit/p2p/QmPTjoYNMLSR8wzeRToMCwokMqCjz1TwBTdbJq7nX4TsWN

so how to use the libp2p-circuit ? or any other examples?
final i get these fixed by set
relayaddr, err := ma.NewMultiaddr("/p2p/"+h2.ID().Pretty()+"/p2p-circuit/p2p/"+ h3.ID().Pretty())

but when i write my own relay code and run it in 3 terminal i got error like this
length greater than remaining number of bytes in buffer
i dont know how to fix it ,
which raw code is in https://github.com/multiformats/multihash

d


Fix Issue 16136: added std.ascii.isNumber

I know it's a one liner, but std.ascii should have equivalents for most of the std.uni functions.

What about renaming isDigit instead?

I can't believe I didn't see that function

I can't believe I didn't see that function

That's why proper naming is of great importance :)


Adds golang server example

r? @adreyfus-stripe @ctrudeau-stripe @tomer-stripe

lgtm after that one comment. thank you!

Thanks for looking, @adreyfus-stripe! updated that one reference.


Improve verification modes error messaging

Basically for following scenario there is a usability issue:

Expect two calls
Two calls are recorded, but one has different arguments.

In this case the message will be Call with identifier xyz was recorded 1 times, but expected at least 2. Would be nice to handle this so the the programmer knows 1 call matches, but there's another one with different arguments.
Somewhat related to #13.

Also closed by the amazing @Eluss in #32 ! Thanks for implementing this!


fix: see below

*CJK Input/Output not worked properly
*Tab indent not worked properly

Can you explain a bit more what problems you were having and what problems this PR fixes?

Test

*CJK means Chinese/Japanese/Korean characters. When these are typed, they cannot display properly.
*Tab problem is, for example, when tabsize is 4 and Tab is typed at cursor location "(1, 2)", cursor does not move to "(1, 5)" but move to "(1, 6)".

Why just not open PR from your tcell to zyedidia's?

Sorry, I forked from zyedidia's tcell and used it because of test purpose. We can use the original zyedidia's tcell.

@zyedidia @imai9999 Micro supports utf8 I/O. I had no problems with CJK I/O provided that the font I was using with my terminal supported it. I was even able to do I/O in Glagolitic as long as my terminal font and input device supported it. ( As is the case on Android. ) I had problems with the Glagolitic font being wider than the cursor, but that's not really something that can be controlled by tcell or micro, afaik, it'd just require me to make sure my font was monospaced and sufficiently wide to accommodate large/wide characters, such as these. As far as I know, even personal use Unicode codepoints should render fine in micro as long as the end font supports it. See here for an example of what I mean. I'd recommend closing this pr and @imai9999 I'd recommend looking into using a new terminal font. I can't recommend any. I don't know any fonts with 100% coverage of Unicode. You may have to switch fonts whenever using CJK. Sorry if this isn't helpful or relevant, I didn't actually look into the changes made in this pr, just the information here in the comments. Here's a link to what my testing looked like on my Android's terminal emulator, though bear in mind I have no knowledge of Chinese or Korean and extremely  limited knowledge of Japanese. ( It's all gibberish in my test image btw. )


Lint

Lint & tested ;)

How was it linted?
It really just looks like whitespace changes according to someone else' style preferences. I have a bit of an eye condition and the whitespace actually makes it easier for me to read code. Less inter-symbol-interference if that means anything to you.

@cfriedt According to PEP8, this is a mistake:
https://www.python.org/dev/peps/pep-0008/#whitespace-in-expressions-and-statements
After PEP8 isn't mandatory but advised for every code.

@cfriedt @cfriedtpv Any news ?

@cfriedt So you plan to be the only maintener/contributor ?
To write open-source Python code without PEP8 is quite useless.
So each contributor has to adapt to the maintener's style (you), when someone codes bugfix and/or features he must know your code style which is non-official and isn't written anywhere...
NP, I fork.


Initial picolibc support

picolibc offers a number of advantages over newlib, some of which include:

alternate "tinystdio" stdio implementation, derived from AVR libc, with a significantly smaller code size and no iprintf/printf dual implementation problem (but this has problems, see below)
standardizing on compiler thread local storage over custom reentrancy structures (albeit this is not necessarily of use to libdragon at this time)
clearer licensing terms (removal of non-BSD-license-compatible source code)

This pull request provides initial support for picolibc, as an opt-in environment variable N64_USE_PICOLIBC=true when building the toolchain. It also provides, separately, an environment variable N64_USE_PICOLIBC_TINYSTDIO=true to enable the alternate stdio implementation. Without these environment variables, newlib is used as before, with no changes.
TODO:

[x] picolibc does not yet support the mips64 CPU family addressed here: https://github.com/picolibc/picolibc/pull/730
[x] ~~tinystdio does not implement funopen(), which precludes supporting streaming compressed assets at this time. ~~ picolibc's tinystdio now implements funopen()!
[x] tinystdio does not implement vasnprintf(), which precludes the stack/heap implementation of rdpq_text_printf. However, this is not necessary; it's solely an optimization strategy, and vasprintf() works just fine.

Of course, compiling with tinystdio disabled means the tinystdio-specific issues don't apply.

    [LD] build/test.elf
      text       data        bss      total filename
    111416      67852       2968     182236 build/test.elf
# picolibc (tinystdio off)
# the difference here is at least in large part due to disabling reentrancy - which wasn't used by libdragon anyway, and picolibc standardizes everything on compiler TLS
    [LD] build/test.elf
      text       data        bss      total filename
     94568      67196       3480     165244 build/test.elf
# picolibc (tinystdio on)
    [LD] build/test.elf
      text       data        bss      total filename
     75496      66292       3816     145604 build/test.elf
# picolibc (tinystdio on, minimal integer-only printf)
    [LD] build/test.elf
      text       data        bss      total filename
     69064      65380       3816     138260 build/test.elf

vasnprintf is a newlib extension not even present in glibc; is there some reason vasprintf isn't sufficient for your use?
funopen is coming soon.

There's no reason vasprintf is not sufficient for libdragon's usecase. vasnprintf is solely used for an optimization where a 512-byte stack buffer is allocated first, then a heap allocation is only made in the unlikely event that vsnprintf creates a larger string. This is far from essential.

This is far from essential.

It's a hot code path though. We would have to change that to vsnprintf first and if that fails, use vasprintf, which is a pity because you've to redo the whole formatting. To be fair, the case where it doesn't fit 512 bytes is cold/unlikely so maybe it doesn't matter much.

Makes sense. I think you could pretty easily build an implementation of the vasnprintf API using vsnprintf and vasprintf which did exactly that at modest cost in code space. It seems like using standard interfaces in your application would have some modest readability benefits at least. vasnprintf is not precisely documented in the newlib code, although the semantics seem fairly obvious.

Rebased with the latest preview branch; funopen() support integrated. This means the PR is probably ready for review.

Rebased again.

Just for you, I added https://github.com/picolibc/picolibc/pull/885

Oh, I modified the PR not to require it months ago, but thank you!

I figured the rest of the group would be happier to not have that regression :-)

(also found that I wasn't testing asprintf at all (oops!))

After discussing on the Discord, we'll try to get the PR evaluated and merged in late December, after this year's N64 Jam is over.

Retested. The picolibc advantage is less significant now that libdragon is threaded, but removing _reent does simplify things a little.
    [LD] build/test.elf
      text       data        bss      total filename
    128280      23016       3572     154868 build/test.elf
# picolibc (tinystdio off)
    [LD] build/test.elf
      text       data        bss      total filename
    107768      22356       4104     134228 build/test.elf
# picolibc (tinystdio on)
    [LD] build/test.elf
      text       data        bss      total filename
     86840      21484       4412     112736 build/test.elf
# picolibc (tinystdio on, minimal integer-only printf)
# LDFLAGS += --defsym=vfprintf=__i_vfprintf --defsym=vfscanf=__i_vfscanf
    [LD] build/test.elf
      text       data        bss      total filename
     81896      20644       4412     106952 build/test.elf


Unable to execute

Hi,
I have a Raspbian GNU/Linux 10 (buster) running on my Raspberry Pi 4 with 4 GB RAM.
It runs Docker version 19.03.1, build 74b1e89
Executing "sudo docker run -d -it -e EULA=TRUE -p 25565:25565 -v /mnt/minecraft:/data --name mc itzg/minecraft-server" pulls the most recent version. All fine.
Mountpoint has been created before.
Trying to start it doesn't seem to work. "sudo docker logs mc" unveils a "standard_init_linux.go:211: exec user process caused "exec format error""
What can I do? I'm not that much experienced with docker etc at all.
Any help is appreciated.

Unfortunately I don't yet have a solution in place for non-amd64 builds of the image. Docker Hub supports multi-architecture base images, but I haven't yet figured out how to setup this image's Hub build to leverage those.
Stay turned since I would really like to solve this given that Raspberry Pi 4 and arm64 systems are now better suited for running the Minecraft server.

...clicked too soon. In the meantime, you could try cloning this repo onto your RaspberryPi and performing a docker build, such as:
docker build -t mc-rpi4 .

Then replace itzg/minecraft-server with mc-rpi4 and it should actually work.

Regards, Peter

Okay, tried it with bedrock, and that failed building the docker image following the scheme above

Glad to hear the build approach worked. Unfortunately bedrock won’t work since they don’t use java but rather prebuild a native (x86) binary.

@itzg  I tried running docker build -t mc-rpi4 --build-arg ARCH=armv7 . it and I got
Step 14/32 : RUN easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT}   --var version=1.2.0 --var app=restify --file {{.app}}   --from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz
 ---> Running in fe9ebf1c9bcc
/usr/bin/easy-add: line 1: syntax error: unexpected "("
The command '/bin/sh -c easy-add --var os=${TARGETOS} --var arch=${TARGETARCH}${TARGETVARIANT}   --var version=1.2.0 --var app=restify --file {{.app}}   --from https://github.com/itzg/{{.app}}/releases/download/{{.version}}/{{.app}}_{{.version}}_{{.os}}_{{.arch}}.tar.gz' returned a non-zero code: 2

@KonradLinkowski this issue has been closed. Why are you building the image when I already provide a multiarch tagged image for armv7 support?

I'm getting standard_init_linux.go:211: exec user process caused "exec format error"  in docker logs when trying to run docker run -d -p 25565:25565 --name mc -e EULA=TRUE itzg/minecraft-server so I thought that this is the answer

No problem at all and glad that image works for you. Building non-x86 architectures is surprisingly hard, so have had to setup a specific branch and build setup for it.


Add Xcode 10 builds

error is:
conans.errors.ConanConnectionError: ('Connection aborted.', error("(10054, 'WSAECONNRESET')",))
don't want to waste CI time re-running appveyor, so merging


Install Works, Crash on Bootup

I followed all the instructions and booted up through MO2 as told using the CC version but when I bootup the game it crashes while loading ENB. I highly doubt specs are an issue as this is the setup I have:
Nvidia Geforce RTX 3070
Intel i9-9900k
16 GB RAM
Installed on an M.2 SSD
Since I have no idea what the problem is exactly I feel I may as well post it here, so here's the crash log:
Crash_2022_1_17_11-31-42.txt
I heard there might be something to do with an anti-virus or whatever but other than the standard Windows stuff I don't have anything like that installed so I don't see what the issue could be.

Ok, turns out it was me trying to launch the CC version. The game apparently doesn't register it so I can't use any of it, how would I transfer that over for use?


underscores should not be allowed in domain names

under_score.com appears to validate but shouldn't. issue seems to be here: https://github.com/whotwagner/domain_name_format_validator/blob/main/lib/domain_name_format_validator/validator.rb#L25

some more context for this: https://www.entrust.com/blog/2019/01/removal-of-underscores-from-domain-names/


Add abchoo as Kubeflow member

Hello,
I am adding myself, abchoo, to the kubeflow members list. I've mainly been working on kfserving's multi-model serving.
My contributions are:
https://github.com/kubeflow/kfserving/pull/1302
https://github.com/kubeflow/kfserving/pull/1306
https://github.com/kubeflow/kfserving/pull/1319
https://github.com/kubeflow/kfserving/pull/1339
https://github.com/kubeflow/kfserving/pull/1352
https://github.com/kubeflow/kfserving/pull/1375
My sponsor is @yuzisun

@googlebot I signed it!

/assign @Bobgy

/lgtm
/approve
Thank you for your contributions!
Welcome to Kubeflow!


[Foreign Key] PRAGMA db call does not work

Per you instruction, I tried
basel.database.run("PRAGMA foreign_keys = ON");
I put it in the app.config and it did not work
I tried it in the controller - when the controller loads and also in a function call and both did not work

It should work.

I know it should BUT IT DOES NOT. Have you tried it?

Yes, I use this in a project, and it work fine.

Can you point me to a repo that I could pull down so I can see how you are implementing it?

https://github.com/jayralencar/sqlite-sync.js/blob/master/CHANGELOG.md

That is a DB call. I am trying set the foreign_keys = ON.
I know this works from the sqlite-sync sqlite.js files bit it does not work from a controller in the application itself. Please try this and you will see it does not work


Support for smallDateTime/Datetime in TVP

recreated pr #407

Codecov Report

Merging #435 into dev will decrease coverage by 0.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev     #435      +/-   ##
============================================
- Coverage     46.28%   46.27%   -0.02%
+ Complexity     2200     2195       -5
============================================
  Files           108      108
  Lines         25210    25210
  Branches       4164     4164
============================================
- Hits          11668    11665       -3
- Misses        11511    11520       +9
+ Partials       2031     2025       -6

Flag
Coverage Δ
Complexity Δ

#JDBC41
45.99% <ø> (-0.11%)
2182 <ø> (-8)

#JDBC42
46.18% <ø> (+0.01%)
2194 <ø> (ø)
:arrow_down:

Impacted Files
Coverage Δ
Complexity Δ

...in/java/com/microsoft/sqlserver/jdbc/IOBuffer.java
54.08% <ø> (+0.13%)
0 <0> (ø)
:arrow_down:

...m/microsoft/sqlserver/jdbc/SQLServerDataTable.java
57.25% <ø> (ø)
23 <0> (ø)
:arrow_down:

...ava/com/microsoft/sqlserver/jdbc/ScrollWindow.java
65.85% <0%> (-7.32%)
16% <0%> (-1%)

...va/com/microsoft/sqlserver/jdbc/StreamColumns.java
87.83% <0%> (-1.36%)
21% <0%> (-1%)

...rc/main/java/com/microsoft/sqlserver/jdbc/DDC.java
44.26% <0%> (-0.45%)
104% <0%> (-1%)

...m/microsoft/sqlserver/jdbc/SQLServerResultSet.java
32.49% <0%> (-0.29%)
239% <0%> (-2%)

...rc/main/java/com/microsoft/sqlserver/jdbc/dtv.java
63.44% <0%> (+0.12%)
0% <0%> (ø)
:arrow_down:

...om/microsoft/sqlserver/jdbc/SQLServerBulkCopy.java
52.67% <0%> (+0.12%)
240% <0%> (-1%)
:arrow_down:

...n/java/com/microsoft/sqlserver/jdbc/DataTypes.java
77.79% <0%> (+0.16%)
5% <0%> (+1%)
:arrow_up:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fcd913d...812aecc. Read the comment docs.


Text features extracted by Glove

Thanks for sharing yor work!!!
Could you please provid the .pkl file that using Glove to extract text features on the CMU-MOSI and CMU-MOSEI datasets.
Looking forward to your reply, thanks!

You can find the CMU-MOSI and CMU-MOSEI datasets, which utilize GloVe for text feature extraction, at https://github.com/pliang279/MultiBench.


feat : Draft code changes for multi submodel version support impl part 2

Description

Added usecase handle implementation for multiple submodel support.

Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review:

[x] DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
[x] Copyright and license header are present on all affected files

@almadigabor Please review and approve the PR.


Develop a way to issue warnings about deprecation

Which components does the task require to be changed? (think hard pls)
snforge
Description
This should be implemented (and first designed) as some kind of universal mechanism where we can put deprecation warnings for items that will be disused or moved to somewhere else

Cairo compiler already has a deprecated attribute we can use. It emits compile-time warnings with custom notes and supports Rust-like feature flags system. See LegacyMap example.


API not working

What happened?
When It's clicked, it only shows "loading...",
Background images are working fine but the quote API is not working properly.

How can we reproduce this bug?
Need to use another API that works fine with the code.
Desktop Information (Optional)
No response
Urgency (Optional)
None

I'd like to fix this @whiteHatpro under JWOC.

@whiteHatpro updated ; )


Getting Mongoose Promise deprecation warning when running test

Getting the 'infamous' Mongoose warning when running the test.
DeprecationWarning: Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html
Tried to use mongoose.Promise = require('bluebird') in user.test.js but still getting the warning.

Good point! Didn't know that this project still uses mpromise! Thanks for pointing it out.

Is there an 'easy' fix for this Simon? Looking through the updates, I thought this was fixed by assigning the global ES6 promise to mongoose.Promise

@Ebsy Can you please attach a screenshot for the same ?
Also can you take the latest code before running tests ?

Hi @KunalKapadia, my mistake after all. I updated mongoose to 4.8.1 and got the warning in 'npm test' at the user API POST. if I downgrade back to 4.7.7 as released, I no longer get the warning. There's an issue open on the mongoose repo - https://github.com/Automattic/mongoose/issues/4951 I'll keep an eye on it, thanks. I'll close this now (and I'll try and remember not to upgrade packages before testing next time ;))


Super class in yii2

Please tell me class hierarchy in yii2.
And what is the super class of yii2

Please use the forums for questions, http://www.yiiframework.com/forum/index.php/forum/42-general-discussions/


Updating the Job CRD

Updating the Event field to type array
https://github.com/volcano-sh/volcano/issues/266

/lgtm
/approve


#114 WIP collection cache

TODO:

[x] Build the cache
[x] Get rid of .unwrap() in update_member closures
[x] Load from cache when possible
[x] Update CollectionWatcher dynamically

PR Checklist:

[x] Link to related issue #114
[x] Add changelog entry linking to issue
[x] Add tests
[x] Add tests for authorization
[x] Add tests for changing data, updating index (e.g. remove stuff)
[x] More tests?

So the thing I'm currently struggling with, is performing the cached query. Currently, Collection does not depend onDb, which means that I can't access Db specific methods from Collection. So should I add collection building methods to Storelike?

Having some trouble with conditionally reversing the sled iterator, depending on the sort_desc setting of the user.
    let iter = if q.sort_desc {
        store.members_index.range(start_key..end_key)
    } else {
        store.members_index.range(start_key..end_key).rev()
    };

Fails, because
`if` and `else` have incompatible types
expected type `sled::Iter`
 found struct `std::iter::Rev<sled::Iter>`

So I read about using Box to prevent this, but that also doesn't work:
    let iter = if q.sort_desc {
        Box::new(store.members_index.range(start_key..end_key))
    } else {
        Box::new(store.members_index.range(start_key..end_key).rev())
    };

Fails:
`if` and `else` have incompatible types
expected type `std::boxed::Box<sled::Iter>`
 found struct `std::boxed::Box<std::iter::Rev<sled::Iter>>`rustcE0308

The changing data test is failing, because the cache is not properly invalidated.
We have this logic, that checks an update atom according to query filters that are being watched:
            let should_update = match (&q_filter.property, &q_filter.value) {
                (Some(prop), Some(val)) => prop == &atom.property && val == &atom.value,
                (Some(prop), None) => prop == &atom.property,
                (None, Some(val)) => val == &atom.value,
                // We should not create indexes for Collections that iterate over _all_ resources.
                _ => false,
            };

But... That goes wrong when the sorted value differs from the filtered value. So let's say we have a QueryFilter that filters by is_a human, and sorts by firstname. Let's say we update the firstname. In this case we want the old Atom to be removed from the index.  So in this case the should_update function is called for the new firstname atom. This atom does not contain the is_a property or the human value. We want to update the existing indexed value, but we can't, because we can't perform the is_a human check. We need all the properties of the Resource, or we need a pre-confirmed check which collections the resource matches.
We should first check if the sort_property matches, then we should check if the resource has the query, I think. This check should be memoized, probably.
Maybe the problem is more fundamental. The should_update function cannot decide if a value needs to be removed, knowing only the Atom and the FilterQuery.
Flip it
Say we want to check if an atom will match things. We could first check if the resource matches the query filter. If that is the case, we can continue, and perform the existing checks.

So the previous cache invalidation issue has now been solved by passing the old resource to the cache invalidation logic. It is needed to match the QueryFilters in query_index.
Things are working fine if a commit is only applied once, but for the /commit endpoint, I'm actually applying a Commit twice.

I think atoms for the QueryIndex and the ReferenceIndex might need to be a bit different.
The difficult part is ResourceArrays.
Let's consider I want add a Paragraph to a Document.
This means I need to add one Atom to the Index.
ReferenceIndex

Needs to update that the Paragraph now has an incoming link from the Document
The old paragraph also still needs a reference
It is entirely possible that in this Atom, both paragraphs are modified (and not just one new appended item). Just to make sure, we will first have to delete all references from the old atom, and then create the new references from the new atom.
This will be an unnecessarily costly operation if we only append an item to an array

QueryIndex

If someone has a Query that sorts document by their number of Paragraphs, their query might need to be updated.
Note that in this case, the IndexAtom will not have any references - it should have a count

Solution
I think we need to have two value fields in a QueryAtom: both a Filtervalue and a Sortvalue. Or we should accept that we can't sort by count. Which is also acceptable.

I think I finally know what is causing my last failing test: remove_atom_from_index and add_atom_to_index should get passed a different resource, not the same one. Should update this in the apply method of commits

Tests are green! I think I'll merge this soon. But first, I'll look for some extra things to break, test and fix.


Return Multiple Tokens from callback

I'm trying to implement a Python-like parse that uses indention as scoping by matching a regex against \n[ \t]* then passing it to a special callback:
fn process_indent(lex: &mut Lexer<Token>) -> ? {
    let mut column = 0;

    for ch in lex.slice().chars() {
        match ch {
            ' ' | '\t' => column += 1,
            '\n' => (),
            _ => unreachable!(),
        }
    }

    match usize::cmp(&lex.extras.indention(), &column) {
        Ordering::Equal => Filter::Skip,
        Ordering::Less => {
            lex.extras.indention.push(column);
            // emit indent token
        }
        Ordering::Greater => {
            let mut down = 0;
            loop {
                match usize::cmp(&lex.extras.indention(), &column) {
                    Ordering::Greater => {
                        lex.extras.indention.pop();
                        // emit dedent token
                    },
                    Ordering::Equal => break,
                    Ordering::Less => panic!("Some type of indention error"),
                }
            }
        }
    }
}

however doing so would require the callback to return multiple tokens in some cases. How would I go about do this?

You can put whatever you want inside a token, like:
enum Indent {
    Tabs(usize),
    Spaces(usize),
}

or
struct Indent {
    tabs: usize,
    spaces: usize,
}

Then have a callback that counts the white space inside the matching token, and put it inside it with a callback:
#[regex(..., |lex| /* construct inner Indent here */)]
Indent(Indent)

To be more stack friendly using u32 or maybe even u16s should be enough for Python I reckon.

I'm looking to implement Koka's layout algorithm and this seems to be the most straightforward way to do it. Since on some lexemes it emits two tokens. I could also implement it as an iterator transformer after lexing, but that's much more annoying. The other option I've considered is having the token type be something like
enum Token {
  Single(RealToken),
  ImplicitOpenBrace(RealToken),
  ImplicitCloseBrace(RealToken),
  ImplicitSemicolon(RealToken),
}

But that feels really bad.

Note that you can also have an enum variant that is simply a Vec<Token>, there is no issue with nested structures (a bit like JSON).

@aricursion I don't think you can do that with a simple enum structure. I think the easiest is to have multiple enums, or a callback that takes the current slice, creates a new nested lexer, exhausts it and returns a Vec<Token>.
It's hard to give a concrete example without any specific application in mind.


How to measure depth in RealSense

| Camera Model                       | D435 D455 |
| Operating System & Version |  Win 10 |
| Platform                                 | PC, NVIDIA Jetson  |
Hi, I'm not familiar with RealSense.
Is the depth in RealSense a linear distance? Or is it the distance from the parallel plane?
Datasheet says that Depth is the distance from the parallel plane.
However, from this site, Depth is the linear distance from the camera.
https://tks2.co.jp/2020/02/03/intel-realsense-error/
Which is the correct information?

Hi @Noctstar  The depth pixel value is a measurement of the distance from the parallel plane of the imagers, as described in the data sheet document.

To arrive at the real-world distance measurement in meters, the depth pixel value is multiplied by the depth unit scale value, which has a default value of 0.001 (meters) on the 400 Series cameras.  So if the depth pixel value was 65535 then multiplying that value by the default 400 Series depth unit scale of 0.001 provides a real world distance measurement of 65.535 (meters)
Further information about the processing of stereo depth can be found in Intel's Depth from Stereo guide.
https://github.com/IntelRealSense/librealsense/blob/master/doc/depth-from-stereo.md
The link that you kindly provided is discussing the error factor in depth measurements over distance.  Depth error in RealSense depth measurements starts at around zero at the camera lens and increases linearly over distance.  So this may be where the question about parallel versus linear comes from.  It is discussing the amount of error in the measurement depending on the distance of an observed surface / object from the camera, not the depth calculation process itself.


ZoomTo: When zooming to whole page/spread, include the bleed

Maybe take into account the preview mode — include bleed if visible?

Closed by 89c3308.


Add resource types in outputs and parameters

Fixes: #2246
This change implements functionality for declaring strongly type
parameters and outputs using resource types.
Example:
param storage resource 'Microsoft.Storage/storageAccounts@2020-01-01'

This declares a parameter that can be interacted with as-if it were an
'existing' resource type declaration of the provided type.
In addition you can do the same with outputs:
output out resource 'Microsoft.Storage/storageAccounts@2020-01-01' = foo

or using type inference (outputs):
output out resource = foo

These features together allow you to pass resources across module
boundaries, and as command line parameters (using the resource ID).

This PR implements #2246 with a few caveats that I discussed offline
with one of the maintainers.

It does not include the 'any resource type' that's outlined in the
proposal. It's not clear how that part of the proposal will work
in the future with extensibility.
It does not support extensibility resources currently. To do this we
need to define the semantics of how an extensibility resource can
be serialized (what's the equivalent of .id). This is explicitly
blocked in the first cut and can be added as extensibility is
designed.
Parameter resources cannot be used with the .parent or .scope
properties because it would allow you to bypass scope validation
easily. The set of cases that we could actually provide validation
for these use cases are really limited.

Contributing a feature

[x] I have opened a new issue for the proposal, or commented on an existing one, and ensured that the Bicep maintainers are good with the design of the feature being implemented
[x] I have included "Fixes #{issue_number}" in the PR description, so GitHub can link to the issue and close it when the PR is merged
[x] I have appropriate test coverage of my new feature

Completions behavior for resource types for a parameter is slightly different than a resource definition. In a resource definition, completions show up both with and without ', but with a param foo resource it only shows up with a ':

Fixing this.

subscribing, to do some more testing once these artifacts are generated.

Is this being worked on? Can't wait to get this functionality...

I pushed an update here. This doesn't yet address all of the scenarios that we've discussed (PR description is up to date). The plan I discussed with the maintainers is to merge this as an experimental feature first and then address the gaps in a series of smaller changes.

I know there's still work left but this is awesome! Since this is behind a feature flag, all the comments I added can be addressed in future PRs.


[improve][io] Upgrade Alluxio to 2.8.1

Motivation
Currently, the Alluxio sink depends on Alluxio 2.7.3, which was released around a year ago.
Modifications
This PR upgrades Alluxio to 2.8.1 so that we can ensure the sink works with more recent version of Alluxio.
Verifying this change

[x] Make sure that the change passes the CI checks.

This change is already covered by existing tests, such as AlluxioSinkConfigTest and AlluxioSinkTest.
Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[x] Dependencies (add or upgrade a dependency)

Upgrade Alluxio 2.7.3 to 2.8.1. Its API changed a bit, so the PR addressed it too.

[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository: https://github.com/sekikn/incubator-pulsar/pull/10
The CI above failed on the BROKER_CLIENT_IMPL and BROKER_GROUP_1 tests, but they seem to be unrelated with this change.

Originally I tried to upgrade Alluxio to 2.9.0, which is the latest version at this writing, but Alluxio's metrics refers to the sun.management package since that version and I came across the following error during using it.
2023-01-19T09:47:18,396+0900 [public/default/alluxio-sink-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [public/default/alluxio-sink:0] Uncaught exception in Java Instance
java.lang.IllegalAccessError: class alluxio.metrics.MetricsSystem (in unnamed module @0x8c05339) cannot access class sun.management.ManagementFactoryHelper (in module java.management) because module java.management does not export sun.management to unnamed module @0x8c05339
        at alluxio.metrics.MetricsSystem.getDirectBufferPool(MetricsSystem.java:165) ~[?:?]
        at alluxio.metrics.MetricsSystem.<clinit>(MetricsSystem.java:157) ~[?:?]
        at alluxio.client.file.FileSystemContext.initContext(FileSystemContext.java:316) ~[?:?]
        at alluxio.client.file.FileSystemContext.init(FileSystemContext.java:305) ~[?:?]
        at alluxio.client.file.FileSystemContext.create(FileSystemContext.java:256) ~[?:?]
        at alluxio.client.file.FileSystemContext.create(FileSystemContext.java:225) ~[?:?]
        at alluxio.client.file.FileSystemContext.create(FileSystemContext.java:207) ~[?:?]
        at alluxio.client.file.FileSystem$Factory.create(FileSystem.java:138) ~[?:?]
        at org.apache.pulsar.io.alluxio.sink.AlluxioSink.open(AlluxioSink.java:101) ~[?:?]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:857) ~[org.apache.pulsar-pulsar-functions-instance-2.11.0.jar:2.11.0]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:224) ~[org.apache.pulsar-pulsar-functions-instance-2.11.0.jar:2.11.0]
        at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:259) ~[org.apache.pulsar-pulsar-functions-instance-2.11.0.jar:2.11.0]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]

Probably I have to add the --add-opens java.management/sun.management=ALL-UNNAMED runtime option somewhere, but I'm not sure where it is yet. So let me upgrade it to 2.9.0+ in another PR.

/pulsarbot run-failure-checks

@sekikn Thanks for your investigation!
I'm not quite sure what our policy is to handle such b/w compatibility issues. Generally, client libs will keep it so version bumps are always b/w compatible. But if it's not the case, at least we should add a release note to describe the breaking changes. I ever did it at https://github.com/apache/pulsar/pull/18774 while that version bump doesn't break fundamental functions.


external IP for kafka service

Hi , I need to  use the ip of kafka for my other microservices. but when am deploying by this methos theres no external IP moreover when am trying with localhost everything is working fine. so can anyone help me in understanding how to set up the ext IP of lb for the Kafka here to use it in

Did you look over the values.yml file around the nodePort line?


Add getNetworkClientById

Add getNetworkClientById, a simple getter for the networkClient registry that takes a networkClientId and returns the corresponding networkClient.

[x] TODO Add tests for this method (if we agree it makes sense)

Rename getNetworkClientsById: this name is/was confusing. To me the ById implies you pass in an Id as an arg rather than that the return will be keyed by id. Open to discussion on this.

It's true that there are a few different concepts that have IDs, so I understand how confusion would arise. I am generally a fan of being explicit where needed, but in this case I also believe that it's reasonable to assume that the ID in this case refers to the thing we're getting rather than some other object, as it's the simplest possibility. If there is confusion, then the method signature includes infuraNetworkClientId or customNetworkClientId in the arguments, so one should be able to hover over the method in one's editor and get a sense of how it's intended to be used quickly.


Modified README a bit, included new link

Thanks for the PR :D But do keep in mind this Repository is intended only for first PRs so any of your further PR will be marked as spam..
Thanks again for your contribution :)


[bug] Unable to make ast-grep scan work with ignored rules in VCS

Please read the FAQ for the bug you encountered.

[X] I have read the existing FAQ

⏯ Playground Link
N/A
💻 Code
// Your code here

🙁 Actual behavior
I have a use case where I want to generate some rules outside of VCS (the point being that we want to enforce some import restrictions, with certain file paths excluded according to our own guidelines).
I don't want to check in tons of generated files in git, but if I exclude the rules in git, they're no longer picked up by ast-grep scan.
🙂 Expected behavior
ast-grep scan should not take .gitignore contents into account when discovering rules (only when deciding on which files to run those rules against).
Additional information about the issue
No response

You can use https://ast-grep.github.io/reference/cli/scan.html#no-ignore-file-type

@HerringtonDarkholme I tried --no-ignore=vcs but this still doesn't pick up the rules. It seems this only applies to discovery for the files the rules are running against and not the rules that are being discovered by ast-grep scan.

Ignoring rules based on ignore files is a confusing behaviour IMO (regardless of --no-ignore). The only situation I can think of for having valid rule files outside of VCS is for generated rules like this, which I would expect to be discovered.
Ignoring files when linting / applying these rules does absolutely make sense and is a great feature of this tool.
Even if this is intentionally configurable - it would probably need a separate config option (so that I can pick up rules from .gitignore but not apply rules to files from .gitignore).

The only situation I can think of for having valid rule files outside of VCS is for generated rules like this, which I would expect to be discovered.

It is really a niche use case where rules are not maintained within the VCS. If ast-grep discovers vcs-ignored rules by default, it will imply that ast-grep will traverse dependency directories like node_modules in the repo to find rules. This may significantly slow down the scan process startup.
I need to think more about how to discover rules.
Meanwhile, ast-grep scan -r can also be used to lint files without committing rules into the repo.

Another work around is:

create a new directory with all ast-grep related files there. Use gitignore to ignore the directory.
Use ast-grep scan -c ignored/sgconfig.yml to discover the project.
Linting the whole file directory should not be impacted.

This is used by next.js, I have not tried it myself yet.
https://github.com/vercel/next.js/tree/canary/.config/ast-grep

The reason I encountered this issue in the first place is because I want to identify and error on unused suppressions, which doesn't work if I need to run ast-grep scan more than once. We were using a second run of ast-grep scan with inline rules prior to this.

It is really a niche use case where rules are not maintained within the VCS. If ast-grep discovers vcs-ignored rules by default, it will imply that ast-grep will traverse dependency directories like node_modules in the repo to find rules. This may significantly slow down the scan process startup.

But doesn't the config decide where rule directories are? Does ast-grep by default recursively scan all files looking for rules?

so neither workaround works for me

I believe the unchecked in config directory workaround should work for you? Would you like to create an example repo to illustrate the problem?

I'm mixing checked in (regular)and non checked in (generated from template with substitutions) rules.
Or can I pass multiple config arguments to ast-grep? ie can I do ast-grep scan -c sgconfig.yml -c generated/sgconfig.yml

can I pass multiple config arguments to ast-grep

I think the easiest solution here is calling ast-grep twice with two different configs?

I want to error on unused suppressions, so it doesn't work for my use case

Can you write a custom script to copy checked in code to unchecked in directory?
This is really a niche case I don't want to support.

it will imply that ast-grep will traverse dependency directories like node_modules in the repo to find rules.

And it caused significant performance regression for other use cases.


Allow to run CI on push to ci-branch

This PR allows to run the CI on push to a branch named "ci-*", without needing to open a PR.

This will allow to make CI tests without opening a PR, e.g., for future huggingface-hub releases, future dependency releases (like fsspec, pandas,...)

Note that to build the documentation, we already allow it on push to a branch named "doc-builder*".
See:

#5788

CC: @Wauplin

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Promise.defer(); is obsolete

In your code, you use const deferred = Promise.defer(); to create a new Promise. According to https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Promise.jsm/Deferred this method is obsolete.
You should use the ES6 way: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
new Promise(function(resolve, reject) {
  if (ok) {
    resolve();
  } else {
    reject();
  }
});

@sylvainlap I also read that the Promise constructor is throw safety. I don't know why we used the common deferred anti-pattern... This is a good timing because we have to rewrite the generators to consider our move to Bookshelf.

Hey @sylvainlap, sorry for the lack of communication this last few weeks-- but this was for a good reason. I have some interesting news regarding this issue.
We decided to split the Strapi stack in two different projects. Here is the reason:

Strapi is and always will be a Node.js framework-- nothing more.
The built-in features such as the Studio, the admin panel and the built-in APIs are not part of the framework as we call it. They are management features. Those features will be in another project will release soon.

The Strapi framework and the utils are for two different kind of users. This is why we decided to split the current ecosystem in two different projects. Don't worry: the built-in APIs and admin panel will still be open-source, even for the new project.
The Strapi v1.x.x will no longer be supported due to those changes and we'll encourage everyone to migrate to the v2. The migration process will be quick and smooth, easy as possible.
We'll soon write a blog post about our choices and why we unfortunately can't afford to support the v1 anymore, like we expected.
Since your issue concerns the built-in APIs and not the framework, I decided to close it aiming to keep the issue tracker lean and focus on the framework only.
Also, your enhancement has been made in this new project, thanks a lot for your suggestion. :-)


Consider using hierarchical configurations

Organize the "config.json" in a better way, by using hierarchical configurations instead.
Suggested by @pawel-czyz in #20 Review.

Pydantic's BaseModel can have default values and have hierarchical configuration.
Hydra is Facebook AI Research's solution of maintaining hierarchical configurations.

Used Pydantic were applicable.


Please confirm these before moving forward

[X] I have searched for my issue and not found a work-in-progress/duplicate/resolved issue.
[X] I have not been informed if the issue is resolved in a preview version of the winget client.

Category of the issue
Other
Brief description of your issue
I encountered this error when creating a new application package manifest. It does not allow exclamation marks in the path. However, there is no restriction on this in the matching rules of the schemas.
Steps to reproduce
create manifests
manifests\h\huanghongxun\HelloMinecraft!Launcher\<IP_ADDRESS>\huanghongxun.HelloMinecraft!Launcher.installer.yaml
# Created using wingetcreate <IP_ADDRESS>
# yaml-language-server: $schema=https://aka.ms/winget-manifest.installer.1.2.0.schema.json

PackageIdentifier: huanghongxun.HelloMinecraft!Launcher
PackageVersion: <IP_ADDRESS>
Installers:
- Architecture: x86
  InstallerType: portable
  InstallerUrl: https://github.com/huanghongxun/HMCL/releases/download/v<IP_ADDRESS>/HMCL-<IP_ADDRESS>.exe
  InstallerSha256: EC9DB614C849A315F20B1E43AEBC5E7ADFAB5F3A7AD6DA04A9886BD6E82F28A5
ManifestType: installer
ManifestVersion: 1.2.0

manifests\h\huanghongxun\HelloMinecraft!Launcher\<IP_ADDRESS>\huanghongxun.HelloMinecraft!Launcher.locale.zh-CN.yaml
# Created using wingetcreate <IP_ADDRESS>
# yaml-language-server: $schema=https://aka.ms/winget-manifest.defaultLocale.1.2.0.schema.json

PackageIdentifier: huanghongxun.HelloMinecraft!Launcher
PackageVersion: <IP_ADDRESS>
PackageLocale: zh-CN
Publisher: huanghongxun
PackageName: Hello Minecraft! Launcher
License: GPL-3.0 License
Copyright: Copyright (C) 2021 huangyuhui
ShortDescription: HMCL 是一个免费、开源的 Minecraft 启动器，允许玩家方便快捷地安装、管理、运行游戏。
ManifestType: defaultLocale
ManifestVersion: 1.2.0

manifests\h\huanghongxun\HelloMinecraft!Launcher\<IP_ADDRESS>\huanghongxun.HelloMinecraft!Launcher.yaml
# Created using wingetcreate <IP_ADDRESS>
# yaml-language-server: $schema=https://aka.ms/winget-manifest.version.1.2.0.schema.json

PackageIdentifier: huanghongxun.HelloMinecraft!Launcher
PackageVersion: <IP_ADDRESS>
DefaultLocale: zh-CN
ManifestType: version
ManifestVersion: 1.2.0

winget install
C:\>winget install -m <manifests path>

Actual behavior

Expected behavior
no error
Environment
[winget --info]
Windows 程序包管理器 v1.3.2091
版权所有 (C) Microsoft Corporation。保留所有权利。

Windows: Windows.Desktop v10.0.22000.856
系统体系结构: X64
程序包: Microsoft.DesktopAppInstaller v1.18.2091.0

日志: %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir

链接
----------------------------------------------------------------------------
隐私声明             https://aka.ms/winget-privacy
许可协议             https://aka.ms/winget-license
第三方声明           https://aka.ms/winget-3rdPartyNotice
主页                 https://aka.ms/winget
Windows 应用商店条款 https://www.microsoft.com/en-us/storedocs/terms-of-sale

Screenshots and Logs

Well, the fix would be to change the PackageIdentifier from huanghongxun.HelloMinecraft!Launcher to huanghongxun.HelloMinecraftLauncher as Windows does not allow ! in the Path.
I can't recall but hopefully the pipelines does not allow special characters as that would break alot of stuff.
Should probably be double checked in both WinGet Create and YamlCreate
@denelon @Trenly

@ryfu-msft do we need to look at how we're validating?


"unable to find lua52" android unity build

Hello!
On Android devices, Lua can't start.
06-16 14:08:52.176: E/Unity(28731): Unable to find lua52
06-16 14:08:52.237: I/Unity(28731): DllNotFoundException: lua52
06-16 14:08:52.237: I/Unity(28731):   at (wrapper managed-to-native) KeraLua.NativeMethods:LuaLNewState ()
06-16 14:08:52.237: I/Unity(28731):   at KeraLua.Lua.LuaLNewState () [0x00000] in :0
06-16 14:08:52.237: I/Unity(28731):   at NLua.LuaLib.LuaLNewState () [0x00000] in :0
06-16 14:08:52.237: I/Unity(28731):   at NLua.Lua..ctor () [0x00000] in :0
06-16 14:08:52.237: I/Unity(28731):   at LogicController.InitNLua () [0x00000] in :0
06-16 14:08:52.237: I/Unity(28731):   at LogicController.Awake () [0x00000] in :0
I assume that because I don't have android lua52 library -> lua52.so. Or lua52.a for iOs.
If anyone has this libs could you share it with me or could you explain to me how to make this libs using lua sources? I'm using Unity for Windows with VS 2015.

Install NDK and download NLua/Lua from Github.
Type 'ndk-build' in android_build folder. You will get a 'libs' folder. There are x86/lua52.so, armeabi/lua52.so, armeabi-v7a/lua52.so in it.
Copy this 'libs' folder to 'Assets/Plugins/Android/', like 'Assets/Plugins/Android/libs/x86/lua52.so. x86 version will probably be used in PC Android emulator, and armeabi-v7a in actual hardware phone.
Click each binary and set proper import settings in Inspector.
Then you get proper packages whether you build Windows version or Android version.
The 'official' lua downloaded from lua.org won't work, because NLua/Lua add some other things in lua sources. I tried and logcat said luanet_registeryindex not found. Be sure to check out logcat when something goes wrong.

Was the issue fixed @Bmak ?

Cause "DllNotFoundException: lua52" is still showing on android and ios build


Could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error

When I set extra_players=['ppo2_cnn:right_players=1,policy=gfootball_impala_cnn,checkpoint=./01600']
I get the error"could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error", and the program hang out
but when I set extra_players=None, it works fine
how can I handle this problem?
Many thanks

Could you provide more logs / stacktrace of the failure?
Also maybe explain a bit your setup? Do you have GPU? Was the checkpoint trained on the same machine?

Thank you for yout reply!
gfootball2.0.3
extra_players=['ppo2_cnn:right_players=1,policy=gfootball_impala_cnn,checkpoint=./01600'] in "/gfootball/env/init.py"
run program by using "python3 -m gfootball.examples.run_ppo2 --level=11_vs_11_hard_stochastic --policy=gfootball_impala_cnn --num_envs=16 --nsteps=512 --noptepochs=2 --nminibatches=4 --lr=0.00011879 --ent_coef=0.00155 --gamma=0.997 --max_grad_norm=0.76 --cliprange=0.115  --render=False --load_path=./01600"
program logs are as followed:
/home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.
Logging to ./output
2019-12-13 09:36:23.003283: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-13 09:36:23.399380: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-13 09:36:23.407691: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-13 09:36:23.424141: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1e7de70 executing computations on platform CUDA. Devices:
2019-12-13 09:36:23.424156: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): GeForce GTX 1080 Ti, Compute Capability 6.1
2019-12-13 09:36:23.424161: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (1): GeForce GTX 1080 Ti, Compute Capability 6.1
2019-12-13 09:36:23.568525: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2019-12-13 09:36:23.570676: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1b0f430 executing computations on platform Host. Devices:
2019-12-13 09:36:23.570743: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): ,
2019-12-13 09:36:23.571046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
totalMemory: 10.91GiB freeMemory: 10.43GiB
2019-12-13 09:36:23.571169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 1 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:09:00.0
totalMemory: 10.92GiB freeMemory: 10.77GiB
2019-12-13 09:36:23.575541: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0, 1
2019-12-13 09:36:23.605994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-13 09:36:23.606063: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 1
2019-12-13 09:36:23.606088: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N Y
2019-12-13 09:36:23.606112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 1:   Y N
2019-12-13 09:36:23.606292: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10145 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
2019-12-13 09:36:23.606824: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 10479 MB memory) -> physical GPU (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:09:00.0, compute capability: 6.1)
2019-12-13 09:36:23.878828: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.878824: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879004: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.878841: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879335: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879349: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879363: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879409: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879554: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879878: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879848: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.879875: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.880171: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.880209: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.880508: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.880834: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.880850: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.881113: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.881321: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.881345: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.881632: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.881679: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.878851: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.878854: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.878854: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.884529: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.878827: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.884529: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.884593: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.888664: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.889739: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
2019-12-13 09:36:23.890002: E tensorflow/stream_executor/cuda/cuda_driver.cc:1204] could not retrieve CUDA device count: CUDA_ERROR_NOT_INITIALIZED: initialization error
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/input.py:57: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.

WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /home/cyq/.local/lib/python3.6/site-packages/baselines/common/policies.py:43: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.

It seems that the problem is with tensorflow not being fork safe.
what worked for me is the following hack (all changes are in run_ppo2.py):

replace "import tensorflow as tf"  with
"from absl import app
from absl import flags"
replace "flags = tf.app.flags
FLAGS = tf.app.flags.FLAGS"
with  "FLAGS = flags.FLAGS"
replace "train()" with "app.run(train)"
replace "def train():" with "def train(_):"
move "  vec_env = SubprocVecEnv([
(lambda _i=i: create_single_football_env(_i))
for i in range(FLAGS.num_envs)
], context=None)" to the beginning of "train" function
add "import tensorflow as tf" right after

Basically it just postpones import of tensorflow after subprocesses are created.
This worked for me.
Could you try if that works for you, and if does, maybe we'll merge this change to the master branch.

It does work for me!
Thank you very much!
I also replace "self._sess = tf.Session()" with
"config = tf.ConfigProto()
config.gpu_options.allow_growth = True
tf.Session(config=config)"
in gfootball/env/players/ppo2_cnn.py
otherwise I'll get a "cuda out of memory" error

Thanks for the additional fix!
We'll make sure to include it in the next release


Calling type() on a const segfaults

To reproduce:
var a: int const = 42;
a.type();

Result: Segfault
Expected Result: Not that

This was patched earlier today - pull the latest commit and try again. If the problem persists, feel free to re-open this.


Rebase ENG-8444 splitting main package

Rebase ENG-8444 split main package to grpc-server, http-server, and hardware
Description
rebase changes from PR-34 made by kelly deng
Following are changes:
- main package is split  into three packages: grpc-server, http-server, and hardware.
- hardware package includes all hardware related i.e, defining hardware structs, getting and exporting data from tink/cacher.
Why is this needed

Fixes: #
How Has This Been Tested?
setup sandbox: build hegel image with these changes and able to provision OS.
How are existing users impacted? What migration steps/scripts do we need?

Checklist:
I have:

[ ] updated the documentation and/or roadmap (if required)
[ ] added unit or e2e tests
[ ] provided instructions on how to upgrade

Rebase ENG-8444 split main package to grpc-server, http-server, and hardware.

look fine. I imagine it's "better" than what we currently have. it's quite a large PR though with lots of changing/moving parts.
Just for the record:
It does feel like we are trading some current complexities/issues for new ones that should have been avoided. Especially since this is not just a quick and easy split of the main package. Things like removing globals, context propagation, graceful shutdowns should have been discussed as some of the design philosophy for this refactor.

Agreed. I don't think we'd want a full refactor atm though so I think the end result being the same is good for incremental progress. It would have been better if the PR itself wasn't one huge commit and instead was smaller obviously correct changes.


(#294) Don't enable completion popup if no suggestions are found

#294

@rexim looks good to me! :+1: Thank you for your contribution!


miss 991: Trial of a proposed new international service, shared code

miss 991: Trial of a proposed new international service, shared code,
refer to https://www.itu.int/dms_pub/itu-t/opb/sp/T-SP-E.164C-2011-PDF-E.pdf

This change is

Again: we never change this file manually, it's auto-generated. We're unlikely to support this country calling code as we don't know of any numbers that actually connect/are valid. Please see the FAQ, specifically the "metadata" section and questions about the countries we support (https://github.com/googlei18n/libphonenumber/blob/master/FAQ.md#why-are-bouvet-island-bv-pitcairn-island-pn-antarctica-aq-etc-not-supported)


added chameleon trait for thieving gloves and voice mask

About the PR

By popular request (and since its a really easy change), I've added the chameleon trait for the voice changer and thieving gloves.
Media

Eh screenshot doesn't really show much. But it does work.

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl:

tweak: thieving gloves and voice changer now have the chameleon trait

thieving gloves should really not be chameleon anyways


Added authentication event, fixes #1

As it turns out the authentication callback guarantees optin parameter making this event trivial to detect:
With empty data-ref:
{
	"object": "page",
	"entry": [{
		"id":<PHONE_NUMBER>62304,
		"time":<PHONE_NUMBER>231,
		"messaging": [{
			"sender": {
				"id":<PHONE_NUMBER>691078
			},
			"recipient": {
				"id":<PHONE_NUMBER>62304
			},
			"timestamp":<PHONE_NUMBER>231,
			"optin": {
				"ref": ""
			}
		}]
	}]
}

Coverage decreased (-1.7%) to 98.276% when pulling 74bf8a1f44dda25c31ab506a219adefaeb2a72cc on maciekmm:feature-authentication into 3b8d6ede4654817e52ee2058c4ead3b9f976f56a on remixz:master.

Coverage remained the same at 100.0% when pulling ffdaebafc7170427a070cd6ab9449f762b966b83 on maciekmm:feature-authentication into 3b8d6ede4654817e52ee2058c4ead3b9f976f56a on remixz:master.


Gopass consuming a lot of ressources and being slow

Since I upgraded to gopass 1.6.2, it takes about two minutes to show me the "gopass: Encrypting secret for these recipients:"-dialog when manually editing a secret via gopass set secret. Until then, gopass consumes about 32% CPU and about 1 GB of RAM (see screenshot). I dont think this is the intended behavior.

Gopass version: gopass 1.6.2 (3cd42aa331ac2a65d2e6554d120a701f404ef266) go1.9 linux amd64
GPG Version: gpg (GnuPG) 1.4.20
My system:
OS: elementary OS 0.4.1 loki
Kernel: x86_64 Linux 4.10.0-37-generic
CPU: Intel Core i7-7500U CPU @ 3.5GHz
GPU: Mesa DRI Intel(R) HD Graphics 620 (Kabylake GT2)
RAM: 4960MiB / 7903MiB

I really love the tool, have been looking a while for something like this!

Uh, this may be due to the recipient parsing. Need to investigate.

In this case I was only encrypting for one recipent, maybe this is of interest.

I can't reproduce this, yet.
Could you please provide some more info?

run GOPASS_DEBUG=true gopass set <secret>
run gpg -k | grep pub | wc -l

Of course you can/should mask all sensitive information in there. For 2. a rough number is enough (no need to disclose exact numbers).

1.:
GOPASS_DEBUG=true gopass set <secret>
[DEBUG] Loaded config: &{Root:0xc42007f860 Mounts:map[] Version:1.6.2 XXX:map[]}
[DEBUG] Loaded config from /home/kolaente/.config/gopass/config.yml: &{Root:0xc42007f860 Mounts:map[] Version:1.6.2 XXX:map[]}
[DEBUG] gpg.Decrypt: /usr/bin/gpg2 [/usr/bin/gpg2 --quiet --yes --compress-algo=none --no-encrypt-to --no-auto-check-trustdb --decrypt /home/kolaente/.password-store/<secret>.gpg]
[DEBUG] GetRecipients(web/shodan) - idfile: /home/kolaente/.password-store/.gpg-id
gopass: Encrypting web/shodan for these recipients:
[DEBUG] gpg.listKeys: /usr/bin/gpg2 [/usr/bin/gpg2 --with-colons --with-fingerprint --fixed-list-mode --list-public-keys<PHONE_NUMBER>88A70C6E99FCA4F40E70337AB24C9B]

 - [Key with name]

Do you want to continue? [Y/n]: y
[DEBUG] gpg.Encrypt: /usr/bin/gpg2 [/usr/bin/gpg2 --quiet --yes --compress-algo=none --no-encrypt-to --no-auto-check-trustdb --encrypt --output /home/kolaente/.password-store/<secret>.gpg --trust-model=always --recipient<PHONE_NUMBER>88A70C6E99FCA4F40E70337AB24C9B]
[DEBUG] store.gitAdd: /usr/bin/git [git add --all web/shodan.gpg]
[DEBUG] store.gitDiffIndex: /usr/bin/git [git diff-index --quiet HEAD]
[DEBUG] Output of '/usr/bin/git [git diff-index --quiet HEAD]': ''
[DEBUG] store.gitCommit: /usr/bin/git [git commit -m Save secret to <secret>: Edited with /usr/bin/editor]
[master 92a69ad] Save secret to <secret>: Edited with /usr/bin/editor
 1 file changed, 0 insertions(+), 0 deletions(-)
 rewrite <secret>.gpg (100%)

2.:
3

Maybe because gopass is using gpg2 instead of simply gpg?

@kolaente Unfortunately I haven't been able to reproduce this. Using gpg2 is definitely fine, since most of our users use gpg2 and there have been no other reports like this. Unless you are able to provide further information we will probably not be able to resolve this.

mhhhh... what further information do you need?


add signature example

https://docs.metamask.io/guide/signing-data.html#signing-data-with-metamask

closed


Move a copy of Code of Conduct to #Openseeds

We do not need a copy of the Code of Conduct but a link to the one in the policies GitHub repo. Please create an issue in the Open Seeds GitHub repository about that. Thanks a lot


Enable metrics gathering & data visualisation of benchmark test results

Enable data visualisation of the benchmark test results with appropriate metrics & graphs. While a starting point could be to use simple plotting utilities such as gnuplot, which can derive necessary data (metrics) from the tool-specific log files, the need is to have a single source/webpage with all relevant plots to aid analysis and interpret the results.
This could be achieved using standard tools like grafana with metrics being collected from services like Prometheus

One part of the solution to the above problem will be addressed by the fix to this issue : https://github.com/openebs/openebs/issues/392
This will enable us to get the "storage-side" metrics while a similar capability needs to be added for the "client-side", i.e., benchmark client-container as well


[BUG] PrivateGPT only able to hold 2 files in its context?

Pre-check

[X] I have searched the existing issues and none cover this bug.

Description
I have installed PrivateGPT on my Mac using the following string:
poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-ollama"
running llama3.1
I have uploaded about 7 HTML files full of text hoping to run queries on them.
With RAG mode selected and with all files unselected (so it should be using all of them) it only seems to be able to hold 2 files in its context window at a maximum.  I see this by the sources listed.
All other modes selected also could not use more than 2 files for context.
Steps to Reproduce
Install PrivateGPT with standard params:
poetry install --extras "ui vector-stores-qdrant llms-ollama embeddings-ollama"
Try  to add more than two files via the GUI and run queries on them in RAG mode.
Expected Behavior
Should be querying all files uploaded
Actual Behavior
Only queries maximum 2 files
Environment
iMac 2021 24", 16GB RAM, 256 SSD, MacOS Sonoma
Additional Information
No response
Version
No response
Setup Checklist

[X] Confirm that you have followed the installation instructions in the project’s documentation.
[X] Check that you are using the latest version of the project.
[X] Verify disk space availability for model storage and data processing.
[X] Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

[ ] Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
[ ] Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
[ ] Ensure proper permissions are set for accessing GPU resources.
[ ] Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)

This problem may be related to the limitation of the context window. It is recommended that the development team check the part of the code that handles the file context to ensure that it can support queries for multiple files.
In addition, you should use Servbay to check your development environment, which is better than Docker on Mac and can ensure that there is no compatibility conflict in the environment

same issue, cant hold more than three pdfs. but in terminal it says it ingested the pdfs but neither its visible in ui nor ai replies with its context. [using openai llm and embeddings]

I am sorry but my issue happen to be this https://github.com/zylon-ai/private-gpt/issues/1960 as there was no error so i got confused.


Naming convention for optional feature flags

Generally, Nixpkgs exports flexibility of upstream build system with boolean
flags to corresponding Nix expression, so, for example, Nix expression for
doas(8) starts with following:
{ lib, stdenv, fetchFromGitHub, bison, pam
, withPAM ? true, withTimestamp ? true }:

There are least two ways to represent the fact that particular input is
optional:

Add boolean flag like withPAM in example above, and use include dependency
with something like buildInputs = lib.optional withPAM pam;
Write code to handle input beeing null instead of derivation, with code like
configureFlags = lib.enableFeature (pam != null) "pam".

In cases when upstream build system provides option to enable or disable
functionality that is not tied to external dependency, only boolean flags
approach is possibly; probably this is why it is more popular.
Furthermore, we don't have convention for naming of this flags. Across the
tree, we have "withPAM", "withPAMSupport", "enablePAM" and "usePAM". The same
mess happens for every other flag. You can get idea with something as simple as
$ git grep -i PAM | grep -iE 'with|support|enable|disable'

Substitute PAM with systemd, nls, ssl or alsa.
This inconsistency, among other things, makes writing overlays harder. On
Gentoo, if you wan't whole system without, e.g NLS, you just set global USE
flag and rebuild the world.
On Nix, you can do the same -- you create overlay, but you need to setup 3 or 4
flags, each of them controlling some irregular subset of all packages. This is
not nice user interface, even (especially?) when user is a developer. I
propose that we make things uniform.
Currently, "supportFoo" seems to be the most common convention, used both for
external dependencies and optional internal features, so the simplest solution
would be to convert all enable/with/disable to this convention. Regular
expression can do it with reasonable accuracy.
Alternatively, and what I would prefer instead, is to follow autoconf
convention: --with for external dependencies, --enable for internal. It
involves more work, more changes, and, unlike previous approach, can't be done
tree-wide.
No matter which of these paths we agree on, I am willing to provide patches for
Nix expressions, nixpkgs manual and probably to nixpkgs-hammering.

@infinisil Something for the nixpkgs architecture team?

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/2024-01-09-nixpkgs-architecture-team-meeting-47/38037/1

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/pre-rfc-standardization-of-feature-parameters/38104/1

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/pre-rfc-standardization-of-feature-parameters/38104/47

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/pre-rfc-standardization-of-feature-parameters/38104/59

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/pre-rfc-standardization-of-feature-parameters/38104/69

This is now a proper RFC: https://github.com/NixOS/rfcs/pull/169

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/need-for-coorinated-enforced-boolean-switch-arguments-in-nixpkgs/48948/4

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/best-practices-for-config-flags-to-packages/4064/8


[BugFix] Byte pass through not effect when using fqdn in k8s environment

Fixes #29677
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[x] Yes, this PR will result in a change in behavior.
[ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[x] 2.5
[] 2.4

[FE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[FE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[BE Incremental Coverage Report]
:disappointed: fail : 0 / 6 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exec/pipeline/exchange/exchange_sink_operator.cpp
0
6
00.00%
[141, 174, 175, 176, 177, 178]

[FE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[BE Incremental Coverage Report]
:disappointed: fail : 0 / 6 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exec/pipeline/exchange/exchange_sink_operator.cpp
0
6
00.00%
[141, 174, 175, 176, 177, 178]

@mergifyio rebase


Split out replace into two options

The operations of replacing a list and merging existing resources need to be handled by separate operations. Atm a readManyUpdate doesn't have the ability to do a common operation:

don't replace the list
replace the returned resources

So it'll need to be like:
{
  replaceList: true,
  mergeExisting: false
}

Atm these are both either true or false. I'll release 1.1 with the limited functionality, then update this in 2.0

I can add this in a non-breaking way in 2.x by deprecating the repalce option.

This may be overkill, but there could be a reason to have a separate replaceMeta as well. Consider the following action:
{
  type: actionTypes.READ_RESOURCES_SUCCESS,
  ids: [23, 54],
  resources: [{...}, {...}],
  replace: true
}

I think what people would actually want to replace here are the resources, but not the meta. However, for this action:
{
  type: actionTypes.READ_RESOURCES_SUCCESS,
  resources: [{...}, {...}],
  replace: true
}

I think they'd want to replace the resources and the meta. So replace is being overloaded here in another way, but this time from the perspective of resources and meta.
So maybe replaceList, mergeResources, mergeMeta?

Resolved in resourceful


Fix: image load bug

Need to set onload before set .src.

Codecov Report
Base: 36.50% // Head: 36.46% // Decreases project coverage by -0.04% :warning:

Coverage data is based on head (a7ac158) compared to base (90d4990).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1077      +/-   ##
==========================================
- Coverage   36.50%   36.46%   -0.05%
==========================================
  Files         335      336       +1
  Lines       14487    14506      +19
  Branches     2221     2227       +6
==========================================
+ Hits         5288     5289       +1
- Misses       8622     8641      +19
+ Partials      577      576       -1

Impacted Files
Coverage Δ

packages/loader/src/gltf/GLTFUtil.ts
0.57% <0.00%> (ø)

...-deserialize/resources/texture2D/TextureDecoder.ts
2.04% <0.00%> (ø)

packages/core/src/DisorderedArray.ts
40.00% <0.00%> (-10.00%)
:arrow_down:

packages/loader/src/gltf/parser/SceneParser.ts
1.02% <0.00%> (-0.12%)
:arrow_down:

packages/loader/src/MaterialLoader.ts
2.32% <0.00%> (-0.12%)
:arrow_down:

packages/loader/src/gltf/parser/MaterialParser.ts
1.35% <0.00%> (-0.08%)
:arrow_down:

...oader/src/gltf/extensions/OASIS_materials_remap.ts
33.33% <0.00%> (ø)

packages/math/src/Color.ts
89.28% <0.00%> (+3.57%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Make 'neon.protocol_version = 2' the default, take two

Once all the computes in production have restarted, we can remove protocol version 1 altogether.
See issue #6211.
This was done earlier already in commit 0115fe6cb2, but reverted before it was released to production in commit bbe730d7ca because of issue https://github.com/neondatabase/neon/issues/7692. That issue was fixed in commit 22afaea6e1, so we are ready to change the default again.

I didn't realize that we reverted this on the 'main' branch last week. @andreasscherbaum asked me about the status so I started digging and realized that this wasn't included in today's release.


Decoder decodes custom types with no fields as foreign types on the Erlang target

Basically, instead of a custom type such as Foo being decoded as TCustom("Foo", []) on the erl target, it is decoded as TForeign("Foo"). This must be due to the way custom types are represented in Gleam under the hood. Don't know much about this, so calling in the erl master @giacomocavalieri.

@giacomocavalieri not sure if you've seen this.

So I tried writing a test to reproduce this failing case but couldn't get it to produce a wrong result:
pub fn custom_type_decoding_test() {
  Foo(1, "a", "a")
  |> dynamic.from
  |> decoder.classify
  |> should.equal(
    TCustom("Foo", [
      Positional(value: dynamic.from(1)),
      Labelled(label: "bar", value: dynamic.from("a")),
      Labelled(label: "baz", value: dynamic.from("a")),
    ]),
  )
}

Did you actually use Foo or another type named differently?

The type must have no fields to be decoded as a TForeign. The type you are trying has three fields.

Even with no fields I still get the test working:
// This passes
pub fn custom_type_decoding_test() {
  Wibble
  |> dynamic.from
  |> decoder.classify
  |> should.equal(decoder.TCustom("Wibble", []))
}

What's the type you encountered this behaviour with?

Oh the gleam.toml has target=javascript in it. You'll have to run with target=erlang

Ooh that explains why I couldn't get it to fail 🤦🏻 Sorry for that! Now the test is failing, thanks for your patience 💕

No worries! It's probably my fault in the first place. Don't know why the config is set to JavaScript.