move out install instructions

This commit moves the install instructions out of the README.md which makes the repository more friendly to newbies.
Signed-off-by: Joseph Wang<EMAIL_ADDRESS>

@joequant the reference to CONTRIBUTING.md should be preserved in the main README.md - could you please restore that? I like pulling out the gory details otherwise.

Just fixed up the file.  Also I don't recall the standard process for
getting onto slack documented anywhere.
On Wed, Apr 20, 2016 at 6:42 AM, Christopher Ferris <
<EMAIL_ADDRESS>wrote:

@joequant https://github.com/joequant the reference to CONTRIBUTING.md
should be preserved in the main README.md - could you please restore that?
I like pulling out the gory details otherwise.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hyperledger/fabric/pull/1164#issuecomment-212157592

@joequant https://slack.hyperledger.org/ is signup page - also listed on hyperledger.org/community

@joequant  Sorry, could you rebase and resubmit? Some PRs were merged out of order.
The changes that I'm aware of

There's now an 'Incubation Notice' in the README. I believe that has to stay there.
The location of the binary changed which resulted in some paths being updated from $GOPATH/src/github.com/hyperledger/fabric to $GOPATH/src/github.com/hyperledger/fabric/peer

Rebased and resubmitted.

Thanks. @christo4ferris Could you review please and make sure all the relevant information is where it needs to be? Thanks

I don't agree with the changes: Both files contain inconsistent information. We should wait for @joshhus to address #1125 then point the readers to appropriate section/document.

@binhn IMO this PR is a step in the right direction. Could you give more detail about what inconsistent information the files contain? It seems that the existing install information was just moved from README.md to INSTALL.md which seems more consistent with other open source projects.
In a future PR I would

Include the architecture diagram and a brief description of Fabric in README.md
Move some of the content in INSTALL.md to CONTRIBUTING.md, such as how to add or update Go packages in the vendor director. Only a contributor would ever need to worry about that.

The reason for making the change is that right now the README.md file is
very, very unfriendly.  Usually a newbie will want an overview of the
project before trying to install.
I just copied README.md to INSTALL.md.   It turns out that we'd likely want
to completely rewrite INSTALL.md but that can be done later.  What I would
propose is that we copy INSTALL.md to a wiki page, and then edit that, and
then periodically sync INSTALL.md to the wiki file.
On Thu, Apr 21, 2016 at 10:15 PM, Sheehan Anderson<EMAIL_ADDRESS>
wrote:

@binhn https://github.com/binhn IMO this PR is a step in the right
direction. Could you give more detail about what inconsistent information
the files contain? It seems that the existing install information was just
moved from README.md to INSTALL.md which seems more consistent with other
open source projects.
In a future PR I would

Include the architecture diagram and a brief description of Fabric in
README.md
Move some of the content in INSTALL.md to CONTRIBUTING.md, such as how
to add or update Go packages in the vendor director. Only a contributor
would ever need to worry about that.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hyperledger/fabric/pull/1164#issuecomment-212938071

@srderson @joequant For example, the readme contains more explanation of the hyperledger project than about the fabric project, which is what this readme is supposed to describe. The hyperledger project is described in hyperledger/hyperledger, and so no reasons to repeat it here. Also the information about incubation has been described on the repo description.
The install file repeats the incubation paragraph (why?) and contains sections on communication, contribution, etc...
So my opinion as expressed above: Let the ID folks do this in a proper way, and we have an issue to do that in this sprint. @joequant could collaborate with @joshhus on the effort.

I'm planning on onboarding about a half dozen student interns this summer,
and the problem is that you need to move most of the INSTALL out of the
README and then work on the INSTALL.md separately.  Also there will be
people who are working on chaincode that will be working in alternative dev
environments.
Putting the INSTALL.md in the README is a bad idea since the information in
INSTALL is going to get very quickly out of date as the code changes and
people come up with different and alternative ways of installing the code.
For example, for the people I'm working with should completely ignore the
current install instructions since they will be working on the docker image
directly.
The readme needs to be somewhat because it's the first thing that people
see when they land on the hyperledger/fabric github site, and it's likely
that they will get here without directly seeing the hyperledger README.md.
I was asked to include the incubation information.
Yes we need to work on the INSTALL.md and the README.md, but the point of
the commit is that the installation instructions should not be in the
README.  The README should just have a list of the major resources so that
people can ask questions when the install doesn't work.  If you just put
the INSTALL inside the README, you are going to have a lot of confused
people when the follow the instructions and they don't work.
On Fri, Apr 22, 2016 at 6:56 AM, Binh Q Nguyen<EMAIL_ADDRESS>wrote:

@srderson https://github.com/srderson @joequant
https://github.com/joequant For example, the readme contains more
explanation of the hyperledger project than about the fabric project, which
is what this readme is supposed to describe. The hyperledger project is
described in hyperledger/hyperledger, and so no reasons to repeat it here.
Also the information about incubation has been described on the repo
description.
The install file repeats the incubation paragraph (why?) and contains
sections on communication, contribution, etc...
So my opinion as expressed above: Let the ID folks do this in a proper
way, and we have an issue to do that in this sprint. @joequant
https://github.com/joequant could collaborate with @joshhus
https://github.com/joshhus on the effort.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hyperledger/fabric/pull/1164#issuecomment-213151020

The other thing is that the main source of INSTALL instructions should not
be in a file at all.  The instructions that anyone with github access uses
to install should be a link to the wiki, that way people trying to install
can interactively modify/extend/edit the installation instructions.  The
only reason for an INSTALL.md file is so that who is offline and gets a
tarball can do an install.
One final thing is that friendly documentation is absolutely critical for
this to work.  The reason that fabric is getting more users than any of the
other projects is that fabric happens to be pretty well documented, whereas
the poor documentation in the other parts of hyperledger make it impossible
for a newbie to get started doing anything substantial.
The README should be a link to the major resources (what is this software
and where do you go for help).  That information is unlikely to change
quickly or become out of date.  The install documentation is likely to lag
behind the code base, and the reason that you want to link people to the
slack list is so that people on slack can tell newbies to ignore parts of
the INSTALL.md file.
On Fri, Apr 22, 2016 at 9:24 AM, Joseph Wang<EMAIL_ADDRESS>wrote:

I'm planning on onboarding about a half dozen student interns this summer,
and the problem is that you need to move most of the INSTALL out of the
README and then work on the INSTALL.md separately.  Also there will be
people who are working on chaincode that will be working in alternative dev
environments.
Putting the INSTALL.md in the README is a bad idea since the information
in INSTALL is going to get very quickly out of date as the code changes and
people come up with different and alternative ways of installing the code.
For example, for the people I'm working with should completely ignore the
current install instructions since they will be working on the docker image
directly.
The readme needs to be somewhat because it's the first thing that people
see when they land on the hyperledger/fabric github site, and it's likely
that they will get here without directly seeing the hyperledger README.md.
I was asked to include the incubation information.
Yes we need to work on the INSTALL.md and the README.md, but the point of
the commit is that the installation instructions should not be in the
README.  The README should just have a list of the major resources so that
people can ask questions when the install doesn't work.  If you just put
the INSTALL inside the README, you are going to have a lot of confused
people when the follow the instructions and they don't work.
On Fri, Apr 22, 2016 at 6:56 AM, Binh Q Nguyen<EMAIL_ADDRESS>wrote:

@srderson https://github.com/srderson @joequant
https://github.com/joequant For example, the readme contains more
explanation of the hyperledger project than about the fabric project, which
is what this readme is supposed to describe. The hyperledger project is
described in hyperledger/hyperledger, and so no reasons to repeat it here.
Also the information about incubation has been described on the repo
description.
The install file repeats the incubation paragraph (why?) and contains
sections on communication, contribution, etc...
So my opinion as expressed above: Let the ID folks do this in a proper
way, and we have an issue to do that in this sprint. @joequant
https://github.com/joequant could collaborate with @joshhus
https://github.com/joshhus on the effort.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hyperledger/fabric/pull/1164#issuecomment-213151020

@joequant There're no standards on readme; there're many projects where readme contains install instruction; however, I am with you on this change, just that I can't accept the current contents of either file. Our current plan to attack this is in #1125 . Please voice your opinions and contribution there.
I disagree that the install instruction be a wiki that any one can modify at any time. We need to review and moderate all changes to the fabric. If someone offered up different and improved methods, we should review and accept accordingly.

It's going to be an interesting culture clash between big companies and
small companies :-) :-) :-)
I run a two person trading firm here in Hong Kong selling goods to Africa
and issuing small business loans.  The philosophy of manufacturing here is
"just do it."  So you have a lot of short, agile cycles.  For this and
anything else, I'd go out of business if I went through a review process
and looked for consensus for everything.  Just do it.
Anyway.  I'll just keep a copy of the changes on my fork, and people can
merge whenever there is interest in doing it.
It's going to be cool to see development cultures clash.
On Sat, Apr 23, 2016 at 6:31 AM, Binh Q Nguyen<EMAIL_ADDRESS>wrote:

@joequant https://github.com/joequant There're no standards on readme;
there're many projects where readme contains install instruction; however,
I am with you on this change, just that I can't accept the current contents
of either file. Our current plan to attack this is in #1125
https://github.com/hyperledger/fabric/issues/1125 . Please voice your
opinions and contribution there.
I disagree that the install instruction be a wiki that any one can modify
at any time. We need to review and moderate all changes to the fabric. If
someone offered up different and improved methods, we should review and
accept accordingly.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/hyperledger/fabric/pull/1164#issuecomment-213612387

@joequant I don't think it's culture. To me, it is just to maintain the correctness of the important information. People always have different ideas, so we need some kind of "gatekeeper" to make sure the content is appropriate.  I am asking @joshhus to reach out to you on #1125

@joequant see PR #1253 - does this start to address your concerns?

I believe the "install instructions" that we want moved to a new INSTALL.md file are basically everything (each internal link and associated content) in /fabric/README.md currently. So the plan is to: 1) copy all of that content to a new file, called /fabric/INSTALL.md; 2) remove all of it from /fabric/README.md; and 3) Add links to README.md to all of the main docs in /fabric/ and to the fabric/docs/README.md.
I'll proceed with a joshhus/ fork to do this work; LMK if there are any changes/corrections to this plan.

The changes have been accommodated by https://github.com/hyperledger/fabric/pull/1341


175B model outputting gibberish?

🐛 Bug

To Reproduce
I'm running python3 -m metaseq_cli.interactive_hosted on a DGX machine not from SLURM.
To avoid the SLURM initialization path I'm setting force_distributed=True to hit _infer_single_node_init.
I'm using the defaults from constants.py except:

I commented out --distributed-port (to avoid SLURM init)
MODEL_SHARED_FOLDER = "/home/hlang/opt_175b_consolidated/"
I set CHECKPOINT_LOCAL = os.path.join(MODEL_SHARED_FOLDER, "reshard.pt")

putting "reshard.pt" at the end here seemed very important? That way in load_model_ensemble_and_task this line:

 filename = filename.replace(".pt", suffix + ".pt")

correctly sets the filename to be reshard-model_part-{model_part_number}.pt for each worker.
This is the main part I'm not sure about?

MODEL_SHARED_FOLDER looks like:
(base) [hlang@node001 ~]$ ls -1 /home/hlang/opt_175b_consolidated/
dict.txt
gpt2-merges.txt
gpt2-vocab.json
reshard-model_part-0.pt
reshard-model_part-1.pt
reshard-model_part-2.pt
reshard-model_part-3.pt
reshard-model_part-4.pt
reshard-model_part-5.pt
reshard-model_part-6.pt
reshard-model_part-7.pt

I got the gpt2-* files from assets/ and the dict.txt from the file @stephenroller posted in https://github.com/facebookresearch/metaseq/issues/19.
That all seems to work: I get all the way to:
2022-05-10 03:00:29 | INFO | metaseq.modules.fused_bias_gelu | Done with compiling and loading fused kernels.
2022-05-10 03:00:39 | INFO | metaseq.checkpoint_utils | Done loading state dict
2022-05-10 03:00:39 | INFO | metaseq_cli.interactive | loaded model 0
2022-05-10 03:00:39 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:12 to store for rank: 0
2022-05-10 03:00:47 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:12 with 8 nodes.
2022-05-10 03:00:50 | INFO | metaseq_cli.interactive | Worker engaged! <IP_ADDRESS>:6010
 * Serving Flask app 'interactive_hosted' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off

Running a prompt
With the server up and running, sending:
wget -O- --post-data='{"prompt": "Paris is the capital of ", "max_tokens": "1", "temperature": "0.0"} --header='Content-Type:application/json' localhost:6010/completions

gives the response:
"choices":[{
   "logprobs":{
      "finish_reason":"length",
      "text_offset":[0],
      "token_logprobs":[56.087547302246094],
      "tokens":[" hoped"],
      "top_logprobs":null},
   "text":" hoped"}]

The server logs for this request look pretty benign (should "input" say something else though?):
2022-05-10 03:44:32 | INFO | metaseq.hub_utils | Preparing generator with settings {'_name': None, 'beam': 1, 'nbest': 1, 'max_len_a': 0, 'max_len_b': 8, 'min_len': 7, 'sampling': True, 'sampling_topk': -1, 'sampling_topp': 1.0, 'temperature': 1.0, 'no_seed_provided': False, 'buffer_size': 4194304, 'input': '-'}
2022-05-10 03:44:32 | INFO | metaseq.hub_utils | Executing generation on input tensor size torch.Size([1, 7])
2022-05-10 03:44:32 | INFO | metaseq.hub_utils | Total time: 0.312 seconds; generation time: 0.309
2022-05-10 03:44:32 | INFO | werkzeug | <IP_ADDRESS> - - [10/May/2022 03:44:32] "POST /completions HTTP/1.1" 200 -

Other queries gave similarly weird text:
wget -O- --post-data='{"prompt": "Review: This movie was terrible! Sentiment: ", "max_tokens": "10"} --header='Content-Type:application/json' localhost:6010/completions

yields:
"text":" play little to become heard sure intended reg to seriously"

You need to set content-type to application/json.

@stephenroller I'm already doing that I think, if you scroll right in the wget command. Sorry for the confusing formatting. I get the same behavior if I POST the json using python requests.

Yeah something's wrong. That first logit is impossible.

The trailing space is messing it up a little bit. The following looks more like I expect.
$ wget -O- --post-data='{"prompt": "Paris is the capitol of", "max_tokens": "10", "top_p": 0.0}' --header='Content-Type:application/json' localhost:6010/completions
--2022-05-10 14:17:50--  http://localhost:6010/completions
Resolving localhost (localhost)... <IP_ADDRESS>
Connecting to localhost (localhost)|<IP_ADDRESS>|:6010... connected.
HTTP request sent, awaiting response... 200 OK
Length: 611 [application/json]
Saving to: ‘STDOUT’

-                                                                0%[                                                                                ]       0  --.-KB/s               {"choices":[{"logprobs":{"finish_reason":"length","text_offset":[0,7,8,9,10,18,21,25,29,37],"token_logprobs":[22.454830169677734,-0.8376489877700806,-0.6941707134246826,-2.215855360031128,-2.050241470336914,-0.6309762001037598,-0.025025367736816406,-2.395449161529541,-2.1187829971313477,-0.7812614440917969],"tokens":[" France",".","\n","I"," thought"," it"," was"," the"," capital"," of"],"top_logprobs":null},"text":" France.\nI thought it was the capital of"}],"created":1652192271,"id":"93af812c-8c35-46a0-ba7a-5ae2d90fd3c1","model":"/mnt/scratch/175B/reshard_no_os/reshard.pt","object":"text_completion"}
-                                                              100%[====================================================================================================================================================>]     611  --.-KB/s    in 0s

2022-05-10 14:17:51 (220 MB/s) - written to stdout [611/611]

(Still has the bad logit though)

🧐 With that query I get:
$ wget -O- --post-data='{"prompt": "Paris is the capitol of", "max_tokens": "10", "top_p": 0.0}' --header='Content-Type:application/json' localhost:6010/completions
"choices":[{"logprobs":{"finish_reason":"length","text_offset":[0,4,8,12,16,20,24,28,32,36],"token_logprobs":[68.44757843017578,-2.7269647121429443,-2.7604899406433105,-2.7923669815063477,-2.7724666595458984,-2.7724084854125977,-2.8166656494140625,-2.8043575286865234,-2.8136215209960938,-2.82196044921875],"tokens":[" won"," won"," won"," won"," won"," won"," won"," won"," won"," won"]

I also get the same thing running interactive_cli:
2022-05-10 11:07:23 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:12 with 8 nodes.
Prompt (ctrl-D to end input, ctrl-C to quit):
Paris is the capitol of[Ctrl-D] won won won won won won won won won won hoped won hoped won hoped won hoped won hoped

Can you paste the full logs from the start of the run through the request service?

Here's everything starting from the python -m metaseq_cli.interactive_hosted (I added a few additional logging statements, but otherwise no changes). The request I made was
 wget -O- --post-data='{"prompt": "Paris is the capitol of", "max_tokens": "10", "top_p": 0.0}' --header='Content-Type:application/json' localhost:6010/completions

python3 -m metaseq_cli.interactive_hosted
2022-05-10 11:18:26 | INFO | metaseq_cli.interactive | Local checkpoint copy already exists, skipping copy
INFERRING SINGLE NODE INIT
2022-05-10 11:18:26 | INFO | metaseq.distributed.utils | distributed init (rank 0): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 2): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 2
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 5): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 5
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 3): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 3
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 6): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 6
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 7): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 7
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 4): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 4
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | distributed init (rank 1): tcp://localhost:14839
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 1
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 0
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 0
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 1
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 3
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 6: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 6
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 7: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 4: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 7
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 4
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 2
2022-05-10 11:18:32 | INFO | torch.distributed.distributed_c10d | Rank 5: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
2022-05-10 11:18:32 | INFO | metaseq.distributed.utils | initialized host node001 as rank 5
> initializing tensor model parallel with size 8
> initializing pipeline model parallel with size 1
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:2 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:3 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:3 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:4 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:4 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:5 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:5 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:6 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:6 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:7 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:7 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:8 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:8 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:9 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:9 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:10 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:10 with 8 nodes.
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:11 to store for rank: 0
2022-05-10 11:18:41 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:11 with 8 nodes.
setting local model part number 4
setting local model part number 5
> initializing model parallel cuda seeds on global rank 0, model parallel rank 0, and data parallel rank 0 with model parallel seed: 2719 and data parallel seed: 1
setting local model part number 7
setting local model part number 0
setting local model part number 2
setting local model part number 1
setting local model part number 6
setting local model part number 3
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-5.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-2.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-4.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-7.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-3.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-1.pt
2022-05-10 11:18:45 | INFO | metaseq.tasks.language_modeling | dictionary: 50272 types
2022-05-10 11:18:45 | INFO | metaseq.hub_utils | loading model(s) from /home/hlang/opt_175b_consolidated/reshard.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-0.pt
GOT WORLD SIZE 1
WORLD SIZE BIG ENOUGH; returning
TRYING TORCH LOAD CPU
torch load cpu /home/hlang/opt_175b_consolidated/reshard-model_part-6.pt
2022-05-10 11:18:55 | INFO | metaseq.checkpoint_utils | Done reading from disk
2022-05-10 11:18:56 | INFO | metaseq.modules.fused_bias_gelu | Compiling and loading fused kernels

NOTE: If this hangs here, your megatron fused kernels may be corrupted. This can happen if a previous job is interrupted during a build. In that case, delete the megatron build directory and relaunch training. The megatron build directory is located at: /home/hlang/Megatron-LM/megatron/fused_kernels/build
Detected CUDA files, patching ldflags
Emitting ninja build file /home/hlang/Megatron-LM/megatron/fused_kernels/build/build.ninja...
Building extension module scaled_upper_triang_masked_softmax_cuda...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module scaled_upper_triang_masked_softmax_cuda...
Detected CUDA files, patching ldflags
Emitting ninja build file /home/hlang/Megatron-LM/megatron/fused_kernels/build/build.ninja...
Building extension module scaled_masked_softmax_cuda...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module scaled_masked_softmax_cuda...
Detected CUDA files, patching ldflags
Emitting ninja build file /home/hlang/Megatron-LM/megatron/fused_kernels/build/build.ninja...
Building extension module fused_mix_prec_layer_norm_cuda...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module fused_mix_prec_layer_norm_cuda...
2022-05-10 11:19:03 | INFO | metaseq.modules.fused_bias_gelu | Done with compiling and loading fused kernels.
2022-05-10 11:19:11 | INFO | metaseq.checkpoint_utils | Done loading state dict
2022-05-10 11:19:12 | INFO | metaseq_cli.interactive | loaded model 0
2022-05-10 11:19:12 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:12 to store for rank: 0
2022-05-10 11:19:12 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:12 with 8 nodes.
2022-05-10 11:19:16 | INFO | metaseq_cli.interactive | Worker engaged! <IP_ADDRESS>:6010
 * Serving Flask app 'interactive_hosted' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
2022-05-10 11:19:16 | WARNING | werkzeug |  * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
2022-05-10 11:19:16 | INFO | werkzeug |  * Running on http://<IP_ADDRESS>:6010/ (Press CTRL+C to quit)
REQUEST_OBJECT: {'input': [32826, 16, 5, 2927, 20884, 9], 'max_tokens': 10, 'top_p': 0.0, 'temperature': 1.0, 'n': 1}
2022-05-10 11:22:20 | INFO | metaseq.hub_utils | Preparing generator with settings {'_name': None, 'beam': 1, 'nbest': 1, 'max_len_a': 0, 'max_len_b': 17, 'min_len': 7, 'sampling': False, 'sampling_topk': -1, 'sampling_topp': -1, 'temperature': 1.0, 'no_seed_provided': False, 'buffer_size': 4194304, 'input': '-'}
2022-05-10 11:22:20 | INFO | metaseq.hub_utils | Executing generation on input tensor size torch.Size([1, 7])
2022-05-10 11:22:21 | INFO | metaseq.hub_utils | Total time: 1.198 seconds; generation time: 1.186
2022-05-10 11:22:21 | INFO | werkzeug | <IP_ADDRESS> - - [10/May/2022 11:22:21] "POST /completions HTTP/1.1" 200 -

Also forgot to mention:
I consolidated the 992 parts using the no-slurm script:
bash metaseq/scripts/reshard_mp_launch_no_slurm.sh <directory_where_all_the_shards_are>/checkpoint_last <output_dir>/ 8 1

It doesn't really look like reshard_mp.py requires any special envvars from slurm, but this is another difference between our setups I think?

Let's check MD5SUM:
934def0c596e01dfb849fa65b73e01aa  dict.txt
75a37753dd7a28a2c5df80c28bf06e4e  gpt2-merges.txt
cf410ee085c5c69c957bb1f6d8456596  gpt2-vocab.json
7e71cb65c4be784aa0b2889ac6039ee8  reshard-model_part-0.pt
c8123da04f2c25a9026ea3224d5d5022  reshard-model_part-1.pt
45e5d10896382e5bc4a7064fcafd2b1e  reshard-model_part-2.pt
abb7296c4d2fc17420b84ca74fc3ce64  reshard-model_part-3.pt
05dcc7ac6046f4d3f90b3d1068e6da15  reshard-model_part-4.pt
d24dd334019060ce1ee7e625fcf6b4bd  reshard-model_part-5.pt
fb1615ce0bbe89cc717f3e5079ee2655  reshard-model_part-6.pt
2f3124432d2dbc6aebfca06be4b791c2  reshard-model_part-7.pt

looks identical 🧐 🤔
(base) [hlang@node001 opt_175b_consolidated]$ md5sum *
934def0c596e01dfb849fa65b73e01aa  dict.txt
75a37753dd7a28a2c5df80c28bf06e4e  gpt2-merges.txt
cf410ee085c5c69c957bb1f6d8456596  gpt2-vocab.json
7e71cb65c4be784aa0b2889ac6039ee8  reshard-model_part-0.pt
c8123da04f2c25a9026ea3224d5d5022  reshard-model_part-1.pt
45e5d10896382e5bc4a7064fcafd2b1e  reshard-model_part-2.pt
abb7296c4d2fc17420b84ca74fc3ce64  reshard-model_part-3.pt
05dcc7ac6046f4d3f90b3d1068e6da15  reshard-model_part-4.pt
8a1e39a98d86b41dc73f07c645990079  reshard-model_part-5.pt
fb1615ce0bbe89cc717f3e5079ee2655  reshard-model_part-6.pt
2f3124432d2dbc6aebfca06be4b791c2  reshard-model_part-7.pt

@stephenroller I'll redownload the *_part-5-shard* files and re-run the merge script and report back.

@stephenroller ok, that was the problem. Part 5 download got corrupted I guess. Now I match your md5sum and I get the same output as you for "Paris is the capitol of". Should we be worried that the first logprob is positive?

Yes, that's a separate bug. I'll file an issue.


re-align selflink to kube

reverts https://github.com/openshift/api/pull/825 to realign to kubernetes.

/hold cancel
There may be a consideration of re-disabling this, but let's see how much more we can push to change with some advance notice.  These are gone by default in kube.

/lgtm


Inconsistent metadata keys when using S3 as input

On 3.47.0, user defined metadata from S3 can't be referenced or deleted when using lowercase metadata keys.
Steps to reproduce:

Upload an object to S3 with user metadata, e.g. x-amz-meta-testkey
Setup benthos with S3 as input
Try to delete the metadata using bloblang or the metadata operator, e.g. meta testkey = deleted()
Send the object to an S3 output

Expected result:
testkey is removed
Actual result:
testkey is preserved
The same scenario applies when e.g. overwriting metadata, for example meta testkey = "overwrite" will just append overwrite to the existing metadata. What works however is a key with a capital letter, e.g. meta Testkey = deleted().
Can this be improved? Thanks

Hey @elundblad, I believe this is because in AWS the metadata keys are case insensitive, and when they're received they are changed to be capitalized for some reason. I'm not sure if there's anything I can do to stop that without breaking changes, but you can rename the metadata keys to force them into lowercase with the following mapping:
input:
  aws_s3:
    bucket: TODO
  processors:
    - bloblang: 'meta = meta().map_each_key(key -> key.lowercase())'

I'll mark this as a documentation issue as it should be made clear in the input docs.

Updated docs: https://github.com/Jeffail/benthos/commit/2b7e7eacf6296403875662a0665595b1bcd57f43


Various fixes for managed transport

Resolves some known issues:

ssh.Dial hangs indefinitely
HTTP leaked connections
SSH leaked connections
Intermittent SSH errors
Panic when closing SSH connections

And also improve overall observability.

Temporary images with the fixes above:
quay.io/paulinhu/source-controller:v0.22.3-transport@sha256:84538232d2847f31793c14192d98d3e1b4c34ea4a179e1faa9186501afb7ee47

quay.io/paulinhu/image-automation-controller:v0.21.1-transport@sha256:bb0f716179c13add6ba019489b9bbf1e77aa2d0badb6804ffb75401c7a982695


Core ML

Well this project is very good.
I am exciting.
I tried to use this model on ios,
So i tired to convert Core ML Model.
But failed,
Could you provide core ml tools?

Thanks for your interest. I do not intend on extending this to edge devices. I don't think it would a common use case and don't have experience in core ML.


LoadingListItemSpanLookup vs GridLayoutManager.SpanSizeLookup

I want to create different sizes of rows.  when i use GridLayoutManager.SpanSizeLookup i can like this:
 mGridLayout.setSpanSizeLookup(new GridLayoutManager.SpanSizeLookup() {
            @Override
            public int getSpanSize(int position) {

                switch (adapter.getItemViewType(position)) {
                    case TYPE_PAGE_VIEWER:
                        return 1;
                    default:
                        return 2;
                }

            }
        });

but with the one in your library it does not return position so i cant have different row sizes.
the one in your library looks like this:
setLoadingListItemSpanSizeLookup(new LoadingListItemSpanLookup() {
                    @Override
                    public int getSpanSize() {
                        return 2; //i want to change this dynamically but no position is passed in  like googles
                    }
                })

my goal is to have a gridlayout that uses your library but that i can have different size rows . for example first row had 3 items but 5th row can have 5 items.  etc.

i checked the source code, it looks like your wrapping around the gridlayoutmanager one.  so it should be ok.  lets close this.


add a script for api update

related to eclipse-xpanse/xpanse#945

@iskey thanks for this. Would it make sense to rename the file as generate_api_client.sh? We can also mention it in the readme file.

@iskey thanks for this. Would it make sense to rename the file as generate_api_client.sh? We can also mention it in the readme file.

sure, let me do this.


SlackReader exceeding rate limit too fast

From my understanding:
                result_messages.extend(                     self._read_message(channel_id, message["ts"])                     for message in conversation_history                 )
The above section read every message/reply per message history and so rate limit is reached very fast

duplicated of https://github.com/emptycrown/llama-hub/issues/269
let's move to there since there are already few ideas to solve


Display Readme on integration/plugin page

Is your feature request related to a problem? Please describe.
Having to click through to the repository to learn more about plugins that I may be interested in seems unnecessary.
Describe the solution you'd like
I would like for the contents of the plugin's readme to be displayed on the store page when you click on the plugin.
Describe alternatives you've considered
This could be hidden by default. This way, the page is still concise, but you could click something like a 'More Info' button that would render the readme.

The developer can add an "info.md" file to show more info https://custom-components.github.io/hacs/developer/general/#enhance-the-experience

but you could click something like a 'More Info' button that would render the readme.

This would effectively be the same as the "REPOSITORY" button (That also render the README, and give you other info as well.)

Thanks. The info.md basically does what I am looking for. Too bad virtually no one seems to be using it.


GAX API raises AttributeError during 'publish'

/cc @bjwatson
======================================================================
ERROR: test_message_pull_mode_e2e (pubsub.TestPubsub)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tseaver/projects/agendaless/Google/src/gcloud-python/system_tests/pubsub.py", line 157, in test_message_pull_mode_e2e
    topic.publish(MESSAGE_1, extra=EXTRA_1)
  File "/home/tseaver/projects/agendaless/Google/src/gcloud-python/gcloud/pubsub/topic.py", line 246, in publish
    message_ids = api.topic_publish(self.full_name, [message_data])
  File "/home/tseaver/projects/agendaless/Google/src/gcloud-python/gcloud/pubsub/_gax.py", line 171, in topic_publish
    event.wait()
AttributeError: 'exceptions.AttributeError' object has no attribute 'message_ids'

To reproduce:
$ git clone git@github.com:tseaver/gcloud
$ cd gcloud
$ git checkout 1855-moar_gax_paging_fixes
$ tox -e system-tests --notest
$ PYTHONPATH= .tox/system-tests/bin/python system_tests/run_system_test.py --package=pubsub

Seems like the PublishResponse object is being returned without message_ids nor an exception.
Have you gotten the API publish(...) call working in any context? IOW, is this a problem is general, or a problem in this specific context?

@bjwatson I had an earlier error in my system test masking this problem, so I have never actually seen publish() succeed with GAX.

@bjwatson One thing different from the other Pubsub APIs (which are working AFAICS, except pull() which is masked by this one) is that push() gets bundled:  the AttributeError gets set by the code which unpacks the response during event.wait().

@bjwatson could it be an issue that the event unpacking code isn't handling repeated elements (e.g., PublishResponse.message_ids) properly?

@bjwatson To reproduce (updated):
$ git clone git@github.com:GoogleCloudPlatform/gcloud
$ cd gcloud
$ tox -e system-tests --notest
$ GCLOUD_ENABLE_GAX=1 PYTHONPATH= .tox/system-tests/bin/python system_tests/run_system_test.py --package=pubsub

cc @tbetbetbe

Hi @tseaver. Sorry for the delay; I was looking into a few other things. I'll try what you suggested and let you know what I find.

I have successfully reproduced this issue on my workstation and am looking into it.

So what I've discovered so far is that event.result is assigned to AttributeError: Assignment not allowed to repeated field "message_ids" in protocol message object.
Based on code analysis, I believe this is being thrown by https://github.com/googleapis/gax-python/blob/master/google/gax/bundling.py#L193. I'm not sure why we haven't discovered this before. Is this the first time the bundle_descriptor.subresponse_field is a repeated field, or should that always be the case? (I don't understand bundling that well, yet)
The bundling configuration for the publish(...) call is:
    bundling:
      thresholds:
        element_count_threshold: 10
        element_count_limit: 1000 # TO BE REMOVED LATER
        request_byte_threshold: 1024 # 1 Kb
        request_byte_limit: 10485760 # TO BE REMOVED LATER
        delay_threshold_millis: 10
      bundle_descriptor:
        bundled_field: messages
        discriminator_fields:
        - topic
        subresponse_field: message_ids

I'm about to sign off for the night. @tbetbetbe Do you have some time to look into this during your morning shift?

@tseaver This is a bug in gax-python. I created issue https://github.com/googleapis/gax-python/issues/110 to track it, and plan to fix it today.

@bjwatson Thanks very much!

@tseaver No problem. I have a fix in code review now: https://github.com/googleapis/gax-python/pull/111
Once this is merged, I will publish new versions of gax-python, pubsub, and logging. And then I will verify that this issue is closed. This should be finished tomorrow.

@tseaver I have fixed this issue with https://pypi.python.org/pypi/google-gax/0.12.1.
Now the behavior I'm seeing is that the test_message_pull_mode_e2e test hangs indefinitely on the second call to publish(...). I'm not sure where in the stack the problem is occurring.
I'm about to wrap up for the day. Can you take a look and see what you think?

I also updated logging, although I have not yet looked into https://github.com/GoogleCloudPlatform/gcloud-python/issues/1889

@tseaver I got caught up in some other things today. Can you or @daspecster look into the indefinite hang on the second call to publish(...), and let me know whether you think it's at the gcloud, gapic, or service layer?

This seems to be a gax-python issue. The hang is happening at https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/pubsub/_gax.py#L170, but only happens for the second call.
I'll look at it some more on Monday, unless @tbetbetbe or @geigerj get to it first.

This is indeed a gax-python issue. The problem is that the delay threshold for bundling is handled using a timer object, but we forget to reset the timer object after it fires the first time... Therefore, this condition will only ever succeed once. I've filed googleapis/gax-python#113 for this.
https://github.com/GoogleCloudPlatform/gcloud-python/blob/master/gcloud/pubsub/_gax.py#L170 is somewhat roundabout... The intended way of achieving the same effect in GAX is just

try:
  result = self._gax_api.publish(topic_path, message_pbs,
      options=CallOptions(is_bundling=False))
  return result.message_ids
except:
  ...

@geigerj Thanks for the pointer!  #1910 disables bundling as you suggest.  We can think about re-enabling it after a fix for https://github.com/googleapis/gax-python/issues/113 is released.


iOS: Delay in tab bar re-appearing after 'back' swipe gesture

Description of issue
When using the swipe gesture on iOS to go back, there is a noticeable delay before the tab bar reappears.
This occurs whenever performing the swipe gesture, be it from an open Event, the Featured events listing, Event filters, Donate page, etc.
Expected behaviour
The tab bar should reappear as soon as the swiping animation has finished.
Steps to reproduce

Open the details page for an event
Swipe right from the left side of the screen

Event is closed, returning to previous page
Delay in tab bar reappearing

OS / Browser / Device model
iPhone 7+ 10.3.1

I've looked into this. It seems like this delay comes from 'react-navigation' library itself. I have built a simple test app using the library documentation testing multiple ways of hiding the tab bar on particular stacked screens. All have resulted in identical delay times after the back gesture has been completed. I'm not sure what can be done to solve this.
source:


timepoints was wrong

Has anyone encountered the same problem as me? After processed with MRIQC 0.15.1, the size_t information in group_bold.tsv was wrong. The time points were inconsistent with the raw data and the preprocessed data.
code:
singularity run --cleanenv -B /seastor:/seastor $HOME/my_images/mriqc-0.15.1.simg
$HOME/bids_dir $HOME/bids_dir/derivatives/mriqc
participant --participant-label $subid
-w $HOME/bids_dir/temporary_files
--n_procs 12
singularity run --cleanenv -B /seastor:/seastor $HOME/my_images/mriqc-0.15.1.simg
$HOME/bids_dir $HOME/bids_dir/derivatives/mriqc
group

The correct number is 226.
I'm not sure if there was something wrong when I running the MRIQC...

I'm not surprised that size_t is smaller than the original size of 226 for several of the subjects, as MRIQC extracts this information after identifying and removing nonsteady states in the beginning of the scan.
I'm more concerned about sub-27 (I think that's the one) and run-2, as you got a length over 226. Could you check that this particular run is not actually 227 long?
I'm closing this issue as it is fairly old and, unless MRIQC is making out that extra volume for the given subject and run, this is completely expected behavior from MRIQC. Please reopen if the given subject and run is not 227-long.
Finally, let me remind everyone that MRIQC is not a preprocessing tool, and intermediate (internal) results of MRIQC should not be used for any purpose other than debugging the tool. In other words, the only outputs of MRIQC that are meant for secondary use are the visual reports and the JSON files containing quality metrics.


Query parameters

There are a couple of issues ( https://github.com/log0ymxm/node-harvest/issues/25 , https://github.com/log0ymxm/node-harvest/issues/11 ) and PR ( https://github.com/log0ymxm/node-harvest/pull/33 , https://github.com/log0ymxm/node-harvest/pull/30 )  related to query parameters
I'm working on a branch to add the functionality once and for all.
i've added a couple of tests and things seems to be working well.
Any comment/help is appreciated

I initially implemented this library to get some basic functionality with the harvest API, and since at the time there was no full node.js clients I stubbed out all the endpoints and a really basic set of tests.
So one of the things that never really got fully implemented were the query parameters, and as you've mentioned they've kind of been ad-hoc implemented by individual contributions. This is needed, I'll see if I can take a look at your work and offer some help. But definitely +1.


Auth handshake fails in 403 (sometimes)

Despite I can successfully login to our Jingo instance, users repeatedly report errors which are reflected by the thrown 403 below.
x.x.x.x - - [22/Jan/2015:08:21:05 +0100] "GET /auth/github HTTP/1.1" 302 0 "http://wiki.transformap.co/login?destination=%2Fwiki%2FTransforMap" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 Iceweasel/31.3.0"
x.x.x.x - - [22/Jan/2015:08:21:06 +0100] "GET /auth/github/callback?code=07548d1af20764a1a354 HTTP/1.1" 302 76 "http://wiki.transformap.co/login?destination=%2Fwiki%2FTransforMap" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 Iceweasel/31.3.0"
x.x.x.x - - [22/Jan/2015:08:21:07 +0100] "GET /auth/done HTTP/1.1" 403 29 "http://wiki.transformap.co/login?destination=%2Fwiki%2FTransforMap" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0 Iceweasel/31.3.0"

I'm running Jingo behind a nginx reverse proxy and proxy_buffering off;.
How can this error happen? Why does the user receive a forbidden status?

An example of a working handshake
x.x.x.x - - [22/Jan/2015:23:43:54 +0100] "GET /auth/github HTTP/1.1" 302 0 "http://wiki.transformap.co/log
in?destination=%2Fwiki%2FTransforMap" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0"
x.x.x.x - - [22/Jan/2015:23:44:07 +0100] "GET /auth/github/callback?code=5c7f95ae6171367dc369 HTTP/1.1" 302 76 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0"
x.x.x.x - - [22/Jan/2015:23:44:08 +0100] "GET /auth/done HTTP/1.1" 302 90 "-" "Mozilla/5.0 (X11; Ubuntu; L
inux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0"

The test is quite simple:
!auth.alone.used && !tools.isAuthorized(res.locals.user.email, app.locals.config.get("authorization").validMatches

Looking at the code it seems that the authorization fails because the user email is not a validMatch. Is it possible?

I start to believe it is an IPv6 issue from the landline there.


test (issue #10711): Test-style error running make test with golangci-lint version 1.44.2

What this PR does / why we need it:
This PR closes #10711. Running make test the results is ok.
Special notes for your reviewer:
If applicable:

[ ] this PR contains documentation
[ ] this PR contains unit tests
[ ] this PR has been tested for backwards compatibility

Hi @jsolana. It looks like there's a merge commit in your commit tree, triggering the DCO bot to fail. Can you please follow the instructions laid out in the failing check?
https://github.com/helm/helm/pull/10714/checks?check_run_id=5409125216
Thanks.

Thanks @bacongobbler :)


Create Enemy sprite

Assumptions / Preconditions:
The user is able to fight off enemies.
Outcomes / Postconditions:
All enemies' sprites will be fully functional and will target the hero.
Normal Scenario:
The user when playing the game will come across enemy sprites that will target the Hero, the user controls.
In the game the user will experience a enemy sprite that will attack the hero if they become too close. There will be one main enemy sprite that is included throughout the entire game.
Error Conditions:
If the enemy sprite isn't animated or has any logical errors it won't target or move around the hero.

This one needs to be more specific . Same comments as #10.

Nice work on this one. I'd still narrow down the specifics even more, but you're on the right track. The scope is more tight than the other issues. You mention multiple sprites, then you mention one sprite - this sounds like it's multiple requirements so you'd want to track with multiple issues.


Add ability to dial API via unix socket

This change adds the ability to route HTTP traffic over a unix domain socket. It does so by reading an optional configuration value, http_unix_socket, containing the path of the socket.
The code here is broken into some smaller changes to (hopefully) aid in its review. We first refactor NewHTTPClient to allow for error returns, and add the new http.RoundTripper implementation returned by httpunix.NewRoundTripper. Next we add configuration support for the new value http_unix_socket. Finally, we read and use this configuration value if present.
In addition to the unit tests around configuration, this change has been used by me internally in my normal work over the last few weeks.
Fixes #3268

Wrong repo


I have an error here.

PS D:\xxx> npm create crx-hmr
Need to install the following packages:
<EMAIL_ADDRESS>Ok to proceed? (y)
TypeError [ERR_INVALID_ARG_TYPE]: The "dest" argument must be of type string or an instance of Buffer or URL. Received undefined
    at cp (node:internal/fs/promises:439:38)
    at file:///C:/Users/xxx/AppData/Local/npm-cache/_npx/bb19c878fe20c9f8/node_modules/create-crx-hmr/bin/cli.js:5:8 {
  code: 'ERR_INVALID_ARG_TYPE'
}

Thanks for opening first issue.
It is window specific error.
Method 1.
First create project folder and  open folder in vs code .
then run npm create crx-hmr@latest
Method 2:
Use git-bash
I hope this error will be fixed. if not, please comment .


UWP with Amazon.Extensions.CognitoAuthentication: NEW_PASSWORD_REQUIRED issue

I'm working on Cognito integration for our UWP solution and using Cognito.Authentication.Extension library; unfortunately, sample code provided here working only if user pool don't have any standard required attribute. Otherwise, this code (from sample)
                    authResponse = await user.RespondToNewPasswordRequiredAsync(new RespondToNewPasswordRequiredRequest()                     {                         SessionID = authResponse.SessionID,                         NewPassword = newPassword                     });                     accessToken = authResponse.AuthenticationResult.AccessToken;
produce HTTP exception (service response) "Invalid attributes given, email is missing" (or name, or any other attribute).
I can't find a way to provide these attributes; also traced Amazon source code, and looks like these attributes are ignored (on client side).
Expected Behavior
Normal flow, new password should be assigned to user, and user become authenticated.
Current Behavior
Exception without proper explanation; I'm unable to change password and get authenticated.
Possible Solution
Please try to cover all your code by unit tests!
Steps to Reproduce (for bugs)
Copy and paste code from here to empty UWP project, create new user pool with (at least) one required attribute, create a new user with temp password and run code.
Your Environment

AWSSDK.Core version used: source code from master branch
Service assembly and version used:
Operating System and version: Windows 10 1803 (OS Build 17134.81)
Visual Studio version: 2017 Pro
Targeted .NET platform: .NET 4.6 (Min version UWP W10 Creators Update 15063)

As a temp workaround, I'll be glad to know what http headers (or other info) your backend is expecting as a required attributes. I tried snippet below, but it doesn't help, same error:
                    RespondToAuthChallengeRequest challengeRequest = new RespondToAuthChallengeRequest
                    {
                        ChallengeResponses = new Dictionary<string, string>
                        {
                            { CognitoConstants.ChlgParamNewPassword, newPassword},
                            { CognitoConstants.ChlgParamUsername, userName },
                            { CognitoConstants.ChlgParamSecretHash, Util.GetUserPoolSecretHash(userName, clientId, clientSecret)},
                            { CognitoConstants.ChlgParamRequiredAttrs, "{\"userAttributes.name\":\"SeNS\"}" },
                            { CognitoConstants.ChlgParamUserAttrs, "{\"name\":\"SeNS\"}<EMAIL_ADDRESS>}
                        },
                        Session = authResponse.SessionID,
                        ClientId = clientId,
                        ChallengeName = ChallengeNameType.NEW_PASSWORD_REQUIRED,
                    };

                    var challengeResponse = await provider.RespondToAuthChallengeAsync(challengeRequest);
                    accessToken = challengeResponse.AuthenticationResult.AccessToken;

I'm running into a similar issue with Amazon.Extensions.CognitoAuthentication.  We can get to the point of passing through the login information, respond to the NEW_PASSWORD_REQUIRED challenge, and get an error that we're not passing all the required attributes (name, in this case).  There doesn't seem to be a method by which to pass ChallengeParameters in the response when using the extensions.  We have to use the extensions to perform these tasks because our roles are locked down, and we're not exposing additional auth flows via our app clients.

It looks like cognito was implemented using this dotnet SDK. It is possible that all cognito users is broken.

For anyone else struggling with this, it's possible to complete the Admin non-SRP flow for a user pool that has required attributes by using classes in the AWSSDK.CognitoIdentityProvider nuget package without using the extensions.  You still start the flow by calling user.StartWithAdminNoSrpAuthAsync, and then you'll get the response with ChallengeName = "NEW_PASSWORD_REQUIRED".
For the next step, construct an instance of AdminRespondToAuthChallengeRequest, and include USERNAME, NEW_PASSWORD, SECRET_HASH and the required parameters prefixed with "userAttributes." in the ChallengeResponses dictionary, like this:
var hash = Util.GetUserPoolSecretHash(username, clientId, clientSecret);
var challengeResponses = new Dictionary<string, string>(5)
{
{ "USERNAME", username },
{ "NEW_PASSWORD", newPassword },
{ "SECRET_HASH", hash },
{ "userAttributes.given_name", "Kirk" },
{ "userAttributes.family_name", "Davis" }
};
var respondToAuthChallengeRequest = new AdminRespondToAuthChallengeRequest {
ChallengeName = ChallengeNameType.NEW_PASSWORD_REQUIRED,
ChallengeResponses = challengeResponses,
ClientId = "app-client-id-here",
Session = sessionId-from-prior-response,
UserPoolId = "user-pool-id"
};
var challengeResponse = await provider.AdminRespondToAuthChallengeAsync(respondToAuthChallengeRequest).ConfigureAwait(false);
//challengeResponse will have a populated AuthenticationResult with IdToken, etc.
Hopefully this helps anyone else struggling with this.


[Bug]: All rows of a DB disappear after I click out of the db settings

Contact Details
No response
What happened?
Weird bug, this only applies to one database where I have sub-folders and automatically file moving set up.
The db functions as expected when I first open it -- I can see all my files, status, tasks, etc. I have a "type" column where I specify which folders I want a file moved to.
When I open up the db folder settings and X out of it, all of the rows disappear and I only see an empty database. This happens even if I don't change anything in settings. If I close and re-open the db, everything is there. So this seems like its just a weird viewing issue related to settings.
What platform were you using?
Desktop
Version of the plugin
3.3.3
Relevant log output
No response
Are you using the latest version of the plugin in your Obsidian vault?

[X] I have verified that I am on the latest version

Are you check if there is a similar issue?

[X] I have verified that there are not similar issues

hello @aparente , Could you share the console logs of your obsidian after this happens? a not controlled exception provokes to do not render the database correctly?

will do - I tried to open to console logs but the commands don't do anything - some other folks have had the same issue: https://forum.obsidian.md/t/how-to-access-the-console/16703/6
when I can figure out how to open the logs I'll try to re-produce!


Replaced amputated arm

What
Replaces missing left (their right) arm on Shruggie.
Why
The floating hand with no arm was a little disconcerting when reading through the docs.
Implications
No changes to behavior, documentation only.

Woah, how could I not see this :scream:
Thanks a lot :wink:


feat: configurable OpenAI API endpoint.

Is your feature request related to a problem? Please describe.
Make the API endpoint url configurable so the user can connect other OpenAI-compatible APIs with the web-ui.
Describe the solution you'd like
Make it configurable through environment variables or add a new field in the Settings > Add-ons.
Describe alternatives you've considered
Tried this but it is not open source and self-hostable.
Additional context
Nill.

Would love to see this as well. Supposedly the new Mistral API platform is OpenAI API compatible, so that too would be an option available that I'm personally most interested in right now. In addition to Azure OpenAPI endpoints, LiteLLM, etc.

Actively taking a look! One question I have is, how would users fetch the list of available models? Do other services offer api endpoints equivalent to https://api.openai.com/v1/models?

The Mistral API seems to be very similar (Docs, API spec)

@Flowm, Could you try to check if Mistral api supports the /models api route? Thanks!

I tried it with mistral and anyscale, and both support /models route. I tried several others in the past few days, and all of them supported it, but I have confirmed for these two just now. They have additional fields in the returned JSON, but they all have data.id, data.object, and data.owned_by in common, which should be enough for integrating with OpenAI compatible tools.

@tjbck I'd be happy to give you an API key (with an expiry date of course) for Mistral if it'll help you get this feature completed. You can find my contact info in my profile.

to name one more on the ever growing list of openAI compatible APIs with resonable pricing: together.ai
API-endpoint: https://api.together.xyz/
they offer a $25 entry bonus and prices for 1M token between $0.1 to $0.9.

Hey guys! Just added the custom api endpoint feature to #265 branch. If any of you guys could try it out and confirm that it works, I'll merge it to main! Thanks!

Hey guys! Just added the custom api endpoint feature to #265 branch. If any of you guys could try it out and confirm that it works, I'll merge it to main! Thanks!

@tjbck Loading the model names from the /models endpoint is working:

However, generation is not responding:

I got the same response with all models. Unfortunately the Docker log doesn't show me anything useful, anywhere else I can look or enable debugging?

@tjbck I just tried this branch. The model list loads correctly, but it shows a CORS policy issue in the browser console.
Access to fetch at 'https://api.mistral.ai/v1/chat/completions' from origin 'http://localhost:8080' has been blocked by CORS policy: Request header field x-title is not allowed by Access-Control-Allow-Headers in preflight response.
POST https://api.mistral.ai/v1/chat/completions net::ERR_FAILED
TypeError: Failed to fetch
at window.fetch (start.c756bce8.js:1:1402)
at X (3.92fc93bc.js:4:1895)
at async 3.92fc93bc.js:1:6227
at async Promise.all (index 0)
at async z (3.92fc93bc.js:1:6142)
at async ce (3.92fc93bc.js:7:2070)```

Just a header problem it seems like, I'll remove some fields and let you guys know!

Please try again and let me know!

I tested it for anyscale and mistral models and it is working perfectly now.

I can archive my chatbot-ui forks now 😂

@ahkhan03 @justinh-rahb Thanks a lot! Could not have implemented this feature without you guys 😅

@tjbck hoping to be able to contribute more, you've got the best UI going for Ollama AND ChatGPT/Mistral APIs right now. There's a lot of people getting interested in running local inference for office workers and solutions like this are key enablers. Here's to more in 2024! 🎉🚀

Thank you for implementing this, HUGELY appreciated!
By far the best UI.
I am using it with TabbyAPI (Exllamav2 API), and unfort they set max_tokens to a default of 150.
Would it be possible in some way to get configurable max_tokens in the UI? It could be called num_predict to be correct for the modelfile, and if sending to an external openai use num_tokens instead?
Tried to use modelfile, but can’t set the “FROM” to an external model. This may be a better way of doing it?
FROM model_name:external
Would really appreciate it. Can’t get around 150 tokens for now!

Hey guys! #381 migrates OpenAI API requests to the backend for proper multi-user support, I'd greatly appreciate if any one of you guys can help with the testing OpenAI compatible APIs, Thanks as always!

I tested it with Anyscale and Mistral APIs, and it is working correctly for all users!!

@ahkhan03 Thanks a lot!

Working with LiteLLM proxy for OpenAI, Mistral, Claude, etc. 👌

hello - how did you configure claude as a model? do you have it locally or is there a way to connect to the anthropic apis..?

My endpoint don't support /models api, is there a way to call my endpoint directly? It's openai-compatible.

You can add Claude by setting up litellm https://litellm.vercel.app/docs/proxy/deploy and then in Open WebUI you add your litellm ip/key in connections and it'll pull all your litellm.
Took a bit of tinkering but once you get it setup it works perfectly.

You can add Claude by setting up litellm https://litellm.vercel.app/docs/proxy/deploy and then in Open WebUI you add your litellm ip/key in connections and it'll pull all your litellm.
Took a bit of tinkering but once you get it setup it works perfectly.

Or use our Pipelines server, this is now our preferred method for Azure/Anthropic/Cohere/Google API access and much more.

You can add Claude by setting up litellm https://litellm.vercel.app/docs/proxy/deploy and then in Open WebUI you add your litellm ip/key in connections and it'll pull all your litellm.
Took a bit of tinkering but once you get it setup it works perfectly.

Or use our Pipelines server, this is now our preferred method for Azure/Anthropic/Cohere/Google API access and much more.

Oh I didn't even realize this was an alternative, thanks will check it out.

Thank you for implementing this, HUGELY appreciated!
By far the best UI.
I am using it with TabbyAPI (Exllamav2 API), and unfort they set max_tokens to a default of 150.
Would it be possible in some way to get configurable max_tokens in the UI? It could be called num_predict to be correct for the modelfile, and if sending to an external openai use num_tokens instead?
Edit: Implemented in #322 along with some small fixes. :)

i can use tabbyapi with curl. i want use with openwebui . what is the endpoint. http://ip:8080  and set auth code dont work for me


Enable eager compilation with signature for dppy.kernel

This will fix the seg-fault issue when signature was provided with dppy.kernel. This will close this Issue
Eager compilation is still Just-In-Time compilation. Instead of the compilation happening at the call site it happens at the definition site. We have caching mechanism for already compiled kernels that takes the SYCL queue in account. This
mechanism can be potentially used to recompile a kernel that has been compiled for a different backend (Sycl queue).
Steps taken to fix this:

Make sure when we provide signature, that triggers the eager compilation, we go through the existing caching mechanism. If the Sycl queue is consistent we simply launch the kernel in other cases we recompile for the given queue.

@reazulhoque please merge with main. It will fix black.

This PR does not fix https://github.com/IntelPython/numba-dppy/issues/75. Because the test uses func.specialize(..., gpu_queue) and the issue have another scenario.

@PokhodenkoSA, this PR solves https://github.com/IntelPython/numba-dppy/issues/75. There are multiple tests that needed to be updated, which test are you talking about when you mention func.specialize(..., gpu_queue)?
I have updated tests with signatures that will trigger eager compilation. I asked Numba team specifically if they have tests for eager compilation and they mentioned they do not. I decided to not write specific tests following that conversation.
When I add signature to @dppy.kernel in exisiting test it follows the example I have provided in #75. Tests I have updated to make sure eager compilation is working:

https://github.com/IntelPython/numba-dppy/pull/291/files#diff-b7c075c42e2cbadf69e707181714af75c0c8a705a806e2e2f71345253fc7af73R56
https://github.com/IntelPython/numba-dppy/pull/291/files#diff-b7c075c42e2cbadf69e707181714af75c0c8a705a806e2e2f71345253fc7af73R81
https://github.com/IntelPython/numba-dppy/pull/291/files#diff-b7c075c42e2cbadf69e707181714af75c0c8a705a806e2e2f71345253fc7af73R108

I agree with test changes. Need CI fix and use platform.system().


AOT setup for tsconfig.json conflicts with ng test

Please provide us with the following information:

OS?

Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)

mcOS 10.12.1
Versions.

Please run ng --version. If there's nothing outputted, please run in a Terminal: node --version and paste the result here:

angular-cli: 1.0.0-beta.18
node: 7.0.0
os: darwin x64
Repro steps.

Was this an app that wasn't created using the CLI? What change did you do on your code? etc.

ng create experiment

cd experiment

npm install

As specified by official Ahead-of-time compilation steps guidelines add the following to tsconfig.json:
"files": [
"app/app.module.ts",
"app/main.ts"
],

P.S: I would like to use the same tsconfig.json for both AOT and normal usage, rather than creating a new one for AOT - is that supported ?
The log given by the failure.

Normally this include a stack trace and some more information.

ERROR in [default] /Users/mmc/temp/experiment/src/test.ts:19:2
Cannot find name 'System'.
ERROR in [default] /Users/mmc/temp/experiment/src/test.ts:20:2
Cannot find name 'System'.
Mention any other details that might be useful.
Happens also with later versions of Angular 2 and earlier versions of nodejs.

https://angular.io/docs/ts/latest/cookbook/aot-compiler.html was written with the https://github.com/angular/quickstart base setup in mind.
When using the CLI you don't need to change anything in the base project setup, ng build --aot should suffice. We take care of all code transforms and changes behind the scenes. This feature is experimental though, there's a few bugs.


8.0.0 Release

I believe I captured most of the important updates for this release. If you notice any I missed, or think some should be added, let me know. As of this release, these notes will be visible in the tracker itself.
Release Notes

Multi-Language support for Spanish, German, French, Italian.

Help us translate the tracker through our online discussion thread.

Emulator crash recovery, periodically saves your game and can restore it if a crash occurs
Log Viewer search and filtering
On-screen game play timer
Added "Release Notes" to the Update screen
Fixed an issue with the auto-updater crashing on Bizhawk 2.8
Fixed a few issues when using Quickload to play a different Pokémon game
Fixed the Levitate ability revealing under some conditions when it shouldn't
Fixed a rare issue where "Heals in Bag" momentarily would show 9999
Fixed an issue where Wally losing in the catching tutorial resulted in a Game Over
New UI updates on several screens
If you notice some Tracker Extensions stopped working, please check each one for updates

To coincide with this release, I've also updated the following extensions to be compatible with this release (they were broken otherwise)

Tourney Point Tracker v3.1
Walking Pixel Pokémon v1.6


RFC: preset: partially allow overrides

What this PR does / why we need it:
Change the way how we handle conflcits among presets, parially allowing some overrides.
Some portions of the spec can be overridden - easy to detect compatible or incompatible updates, some others should not be overriden.
Special notes for your reviewer:
Partially addresses https://github.com/kubevirt/kubevirt/issues/1375
Related to https://github.com/kubevirt/kubevirt/pull/1383 - spawned from initial comments
This is a RFC. The PR is reviewable but not complete: I want to see if I can add more tests.
Release note:
Change the way we detect conflicts between presets, partially allowing overlaps between profiles.

after long discussions, turns out we don't want to go in this direction after all


Updates to various 2020 events.

ANZ Serverless Days replaces ServerlessDays MEL and SYD. Web Directions is moving to 2021. API Days is online. ComponentConf is cancelled. GIDS is no longer a local event.

Thanks again for the updates. Much appreciated.


Start one state implementation

This is an initial attempt that I need for the robustness analysis. To complete this, two things are missing form my point of view:

handling self._schur_vectors, self._coarse_T, self._coarse_init_dist, self._coarse_stat_dist. For now, I just set them to None. Running one of the plotting associated plotting functions will however urge me now to compute_metastable_states. Either we create dummy data for them (everything will be 1), or we modify the associated error messages to say 'Compute metastable states first using more than 1 state'
adapt compute_main_states to the one-state case: take the one state and make it the main state (no redistribution).

One more thing: when we use the stationary distribution for root/final_states_probs, the probabilities will be much smaller than usually because the stationary distribution is scaled to sum to one.
Once way to solve this would be to simply scale root/final_states_probs such that the largest value is always 1. What's your opinion?

@michalk8 I'm merging this, should be fine. I adapted the error messages when a user tries to call e.g. plot_coarse_T to tell him/her to use n_states > 1.


poetry config unaffected by action inputs

I'm a novice with GitHub Actions, so let me preface this by saying that this issue could be entirely on my end :)
Anyways, I'm seeing that virtualenvs.in-project is always set to true when I run poetry config --list as a run action despite the default value for the action input virtualenvs-in-project being false. I've also noticed that setting the non-local config in a subsequent action, as per the README, does not actually affect the poetry config. Setting subsequent configs with --local is visible.
I did notice that the version action input is respected because I tested setting this to latest, 1.1.10 and 1.1.9 to check if it would respect that and it did.
Am I doing something wrong here? Taking a stab but perhaps the system-wide poetry config file is not writable?
Thanks!

Thanks for reporting this @willfrey 👏
We might be loading in some of the inputs incorrectly. I only ever install venvs in-project, since they're simpler to cache, so I've never actually tried setting it to false, and I don't think we have a test for it.
This might be related to the new quoting of inputs mentioned in #54.
Wrt. setting config options in subsequent steps - it's entirely possible that I've made a wrongful assumption at one point. If I understand you correctly, and using --local makes this work, maybe we should just update the README 🙂
Would you be interested in submitting a PR for either of these issues?
If not, I'll probably have time to review this over the weekend at some point.

I can fiddle around with it and if I figure it out, I can submit a PR. But if you have time over the weekend, please don't wait for me 🙂
FWIW, the default settings are fine for me and I agree that having the virtualenv inside of the project is easier for caching.
I was just seeing how the inputs would affect the VENV environment variable as a means of understanding GitHub Actions syntax better.
Quick question just for personal clarity: is the content of the scripts directly unused now? It seems like everything is taking place in action.yml.

Yeah no worries, happy someone's experimenting, because I'd never have found out about this myself 😛
The scripts folder is legacy. Made some sub-optimal choices design-wise in v1.0 and v1.1. Those versions curl the config script from this repo's main branch, so removing them will break the action for anyone running those versions 😵💫

I won't be able to submit a PR to fix this issue (since I'm not making much progress 😓) but I will open another PR that fixes a few minor things like updating  LATEST_POETRY to be 1.1.10 and fixing some cosmetic whitespace.

#57 seems to fix the issue with config arguments not being set correctly.
In regards to setting config options in subsequent steps, I tried replicating the issue in https://github.com/snok/install-poetry/runs/3771474449?check_suite_focus=true, but was unable to do so. Do you have a sample of what your action looked like when you saw the issue @willfrey? 🙂

Let me copy that snippet into my little sandbox repository and try it out to see what's going on before I send you on a fool's errand :)

Much appreciated 👏

https://github.com/willfrey/learn-github-actions/runs/3772228198?check_suite_focus=true
Check this guy out. virtualenvs.in-project seems to fail to be set.

Huh, yeah that's odd. Might take another look tomorrow 🙂

So I tried replicating it once again in https://github.com/snok/install-poetry/runs/3775961027?check_suite_focus=true.
Your workflow is
  test-poetry-config:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: 3.9
      - uses<EMAIL_ADDRESS>      - run: poetry config virtualenvs.in-project false
      - run: poetry config --list
      - run: poetry config virtualenvs.in-project true
      - run: poetry config --list

and I set up this:
  test-setting-virtualenvs-in-project:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: ./
      - run: poetry config virtualenvs.in-project
      - run: |
          source .github/scripts/assert.sh
          poetry config virtualenvs.in-project true
          assert_in "true" "$(poetry config virtualenvs.in-project)"
      - run: |
          source .github/scripts/assert.sh
          poetry config virtualenvs.in-project false
          assert_in "false" "$(poetry config virtualenvs.in-project)"
      - run: |
          source .github/scripts/assert.sh
          poetry config virtualenvs.in-project true
          assert_in "true" "$(poetry config virtualenvs.in-project)"

As far as I can tell, they both use the same action version, and they both try to change the same config setting in the same way, with the same syntax; yet, yours failed and mine worked 🤔
To be certain there's nothing weird about this repo (I think debug-mode is turned on for action runs, so thought that might somehow have something to do with it) I did a test in another repo, and that worked too: https://github.com/snok/pep585-upgrade/runs/3775986917?check_suite_focus=true
So not sure what's going on here exactly - do you have any ideas?

Perhaps try yours by naming your repository explicitly instead of ./?

I'm also running the setup Python action before I run your action, so I'm not sure if that matters.

The run at https://github.com/snok/pep585-upgrade/runs/3775986917?check_suite_focus=true does the same 🤔 It runs v1 rather than v1.2 but they should also point to the same release.

Still appears to not be setting it for me on this run. Strange, right?
I also find it curious that the default setting for virtualenvs.in-project is true even though I believe your default is false, as is poetry's default.

Doesn't replicate unfortunately. I created a test-repo with your exact workflow here: https://github.com/sondrelg/issue-55/runs/3795650618?check_suite_focus=true.
Are you running on self-hosted runners or something?

Nope, just the default GitHub ones. I'll try making a new repo. Maybe it's some cache issue?

Did you make any progress on this @willfrey? 🙂 I'll close this for now, since I've not been able to reproduce, but feel free to get back to me with any new info 👍


ENH: Support merging DataFrames on a combination of columns and index levels

[ ] .groupby #5677
[ ] .sort_values #14353, though this is directly in combat with .sort_index and non-explict
[ ] .merge (this issue)

Overview
@jorisvandenbossche
As a part of the Pandas 1.0 goal to "Make the index/column distinction less painful (#5677, #8162)" I propose that the df.merge method support merging DataFrames on a combination of columns and index levels.
This could be accomplished in the API by allowing the on, left_on, and right_on keywords to accept a combination of column names and index level names.  Any index levels that are joined on would be preserved as index levels in the resulting merged DataFrame, while all other index levels would be removed.
This proposal is in the spirit of #5677 for df.groupby  and #14353 for df.sort_values.

+1 on this proposal. As I said in the related issue (https://github.com/pydata/pandas/issues/14353#issuecomment-251911933), would be nice to have a general solution for this, but for now enabling this behaviour for specific functions/keywords is fine for me.
@jreback @wesm @TomAugspurger @shoyer @sinhrks @chris-b1 any concerns of feedback regarding this proposal, before work is done to implement it? (@jmmease you plan to tackle this if OK?)

Yes, if the direction is agreeable I plan to begin tackling this set of issues during the next month or two.
Thanks for the feedback!

I think I'm in favor of all the changes. Thanks for taking them on @jmmease!

Yes, seems like a good idea to me.

We do need to clarify how we will handle conflicting index/column names in a uniform way. For backwards compatibility, I think we need to always check column names before falling back to use index names.

@shoyer Agreed regarding conflict resolution. Thanks for the feedback!


Add DashR MapD demo

Issue for app: #[issue number here]
App pull request

[x] This is a new app
[ ] I am improving an existing app (redesigns/code "makeovers")

About

Playground deployment URL (new version):
Current gallery app URL: https://dash-gallery.plotly.host/dash-mapd-demo/
Python app repository link: https://github.com/plotly/dash-sample-apps/tree/master/apps/dash-mapd-demo

Workflow

[x] I have created a branch in the appropriate monorepo, and the
elements necessary for successful deployment are in place.
[ ] If the app is a redesigned and/or restyled version of an
existing gallery app, I've summarized the changes requested in the
appropriate Streambed issue and confirm that they have been applied.
[x] If the app is on the Dash Gallery portal, I have added a link to
the GitHub repository for the source code in the portal description.
[x] If the app is a reimplementation of a Python gallery app for the
DashR gallery, the app in this PR mimics, as closely as possible,
the style and functionality of the existing app.=
[x] I have removed all Google Analytics code from the app's
assets/ folder.

The pre-review review
I have addressed all of the following questions:

[x] Does everything in my code serve some purpose? (I have removed
any dead and/or irrelevant code.)
[x] Does everything in my code have a clear purpose? (My code is
readable and, where it isn't, it has been commented appropriately.)]
[x] Am I reinventing the wheel? (I have used appropriate packages to
lessen the volume of code that needs to be maintained.)

As per https://github.com/plotly/streambed/issues/13854, I am closing this as the new deployment process is in place.
If you need any changes from this branch, please recreate your changes against master and create a new pull request. Let me know if you need help and I am happy to jump in.


env.js is not created

Describe the bug
Hi, I'm still not entirely out of the woods for Telplin.
The envPath file is not being created after running dotnet perla b.
To Reproduce
See https://github.com/nojaf/telplin
My config does seem to work for dotnet perla s.
Any ideas?

beta 15 includes the changes required for telplin env (and also for fantomas-tools) that should make it work as we'd expect it in telplin to work.
I will make some ergonomic centered changes around the env imports for beta 16/17
Since we've been hitting a lot of issues with Fable's absolute path imports I want to skip that completely, I don't know where my mind has been but we can use import maps for this purpose...
With the following import map:
{
  "imports": {
    "react": "https://esm.sh/react",
    // the rest of the dependencies
    // Add an entry for the env file right here
    "App:Environment": "/env.js"
  }
}

We can just change the import to the following:
[<ImportMember("App:Environment")>]
let SERVER_URL = jsNative

import { SERVER_URL } from "App:Environment"

It will work cleanly, and we can avoid relying in fable's resolution rules I'm not sure if I should make that bare import (App:Environment) configurable or just settle in that naming convention
Anyways I'll open a different issue to track that, I'll close this issue as it should have been resolved, but if it doesn't feel free to let me know and we can re-open this
(by the way as an extra note... beta15 doesn't have dotnet perla s available, I'm waiting on https://github.com/JordanMarr/FSharp.SystemCommandLine/pull/15 to add it back in the proper manner)


ViewPager stucked when switch tab using FAM

ViewPager not working smoothly on tab changed

any update about this issue

any update?


[bug] [macos] physical position of windows and monitors reported incorrectly

Describe the bug
Setup: I have a Macbook Pro hooked up to an external monitor as below. The Macbook is set to "more space" which means it's a 3600x2388 monitor scaled to 1800x1169. Next to it is an external monitor running at 3440x1440.

The laptop screen works fine in isolation: monitor.size() returns PhysicalSize { width: 3600, height: 2338 } and monitor.position() returns PhysicalPosition { x: 0, y: 0 }. Windows inside the monitor report a physical position with x between 0 and 3600. So far so good. It's just using totally unscaled numbers.
But the second monitor reports its position as PhysicalPosition { x: 1800, y: -271 }. This implies it's using the scaled size of the laptop screen to compute the external monitor's offset in physical space:

1800 is the scaled width of the laptop screen, so 1800 is the starting x value the secondary monitor to its right
because the scaled height of the laptop screen is 1169, the secondary monitor is 271 pixels taller and thus starts at -271

So it's being inconsistent: widths unscaled, offsets scaled.
What makes this problematic is that windows inside the second monitor report their physical positions in ways consistent with the second monitor. So a window there has a physical x position of between 1800 and 5200 (1800 + 3440 = 5200).  This makes the total physical space inconsistent, such that:

there are multiple window positions that report the same physical position (e.g. x=2000 corresponds to a position on both monitors)
dragging the window from left to right so that it crosses into the second monitor will cause the physical position to rise until it his the edge of the monitor, then drop sharply down (from 3599 to 1800)
calling set_position() on its current physical position can switch which monitor it's on:

    match event {
        tauri::WindowEvent::Moved(tauri_pos) => { // physical position of moved window on second monitor
            println!("{:?}", tauri_pos); // => PhysicalPosition { x: 1947, y: -12 }

           // now move it to the position it thinks it's already at
         // moves it back onto the first monitor!
          window.set_position(tauri_pos);

In practice, where this came up for me is that I save the positions and current monitor of windows so that when the user reopens the application, the windows reopen in the right places. I check that the saved positions lie within the bounding box of the stored monitor name and then set_position the tauri windows into place. But it often puts them on the wrong monitor. This is because the stored physical position is actually ambiguous, since I recorded it from the position on the secondary monitor but it's equally valid on the laptop monitor and that's where Tauri puts it.
Reproduction
No response
Expected behavior
I think monitors and windows need to be completely consistent about whether physical position corresponds to scaled or unscaled sizes and positions. (I suspect they should mean scaled sizes always, but that may have consequences I don't understand; the more important point is to be consistent.)
Platform and versions
[✔] Environment
    - OS: Mac OS 13.5.2 X64
    ✔ Xcode Command Line Tools: installed
    ✔ rustc: 1.69.0 (84c898d65 2023-04-16)
    ✔ Cargo: 1.69.0 (6e9a83356 2023-04-12)
    ✔ rustup: 1.26.0 (2023-04-05)
    ✔ Rust toolchain: stable-aarch64-apple-darwin (default)
    - node: 20.5.1
    - yarn: 1.22.19
    - npm: 9.8.0

[-] Packages
    - tauri [RUST]: 1.4.1
    - tauri-build [RUST]: 1.4.0
    - wry [RUST]: 0.24.3
    - tao [RUST]: 0.16.2
    - @tauri-apps/api [NPM]: 1.4.0
    - @tauri-apps/cli [NPM]: 1.4.0

[-] App
    - build-type: bundle
    - CSP: default-src blob: data: filesystem: ws: wss: http: https: tauri: asset: customprotocol: 'unsafe-eval' 'unsafe-inline' 'self' img-src: 'self'
    - distDir: ../dist
    - devPath: http://localhost:3000/
    - framework: React
    - bundler: Vite

Stack trace
No response
Additional context
No response

Still not fixed in v2.


Error parsing macro call arguments

The macro call does not properly parse arguments if an argument has parenthesis in it. for instance Duration::from_secs(60).
This appears to be a simple parser error that can be fixed but I can't work on it right now so I'm creating this issue to not forget it.

Closed because this was fixed in 3fa0b79


Update hmm parameters to use dataclasses instead of dicts.

Param dict --> dataclass
This PR changes hmm parameters to be stored in nested dataclasses rather than dictionaries.
This introduces four different types of dataclass, one for initial params, one for transition params, one for emissions params and the top level class which houses an instance of each of the above. Something like:
@chex.dataclass
class InitialParams:
    probs: Float[Array, "state_dim"]

@chex.dataclass
class TransitionsParams:
    transition_matrix: Float[Array, "state_dim state_dim"]

@chex.dataclass
class EmissionsParams:
    means: Float[Array, "state_dim emission_dim"]
    scales: Float[Array, "state_dim emission_dim"]

@chex.dataclass
class Params:
    initial: InitialParams
    transitions: TransitionsParams
    emissions: EmissionsParams

This PR defines the appropriate dataclasses and updates the relevant model/emission objects to use/return a dataclass rather than a nested dictionary object. The naming convention is to prepend "Params" to the relevant class: Params*, e.g. ParamsPoissonHMMEmissions, ParamsPoissonHMM, ParamsStandardHMMInitialState, ...
The fields of a dataclass can be accessed with either 'square-bracket notation' like a dictionary (e.g. params['probs']) or with 'dot notation' (e.g. params.probs). However only the dot notation can be used if the value of the field is being set. Therefore, where necessary, square-bracket access has been replaced by dot access.
to/from unconstrained
To reflect change to using dataclasses, the to_unconstrained and from_unconstrained functions have been updated and their behaviour has slightly changed.
The function to_unconstrained previously returned two dictionaries, unc_params and fixed_params, containing the unconstrained and fixed parameters respectively. It now returns the unconstrained parameters in a dictionary (as before) alongside the original params dataclass which contains all parameters (not just the fixed parameters).
The function from_unconstrained takes a  dictionary of unconstrained parameters (as before) as well as a dataclass containing all of the parameters (orig_params) and for each parameter in unc_params replaces the corresponding leaf in orig_params with the appropriately constrained value from unc_params.
Possible future changes/additions:
At present all the tests in hmm/ are passing in this branch but this change has most likely broken almost all of the hmm notebooks, fixing them is a priority.
Other possible updates include:

Parameter properties are still in a dictionary rather than a dataclass.

The question of how to handle properties is still a bit up in the air and so this can be left for the time being.

Combine Parameter classes where possible.

At present there is some redundancy, e.g.  ParamsLinearRegressionHMM and ParamsLinearAutoregressiveHMM have identical fields and types and should perhaps be combined.

It looks like this has broken some of the lgssm models (two tests failing) I'll try to fix this now.

lgssm parameters have now been similarly updated to nested dataclasses:
@chex.dataclass
class ParamsLGSSMInitial:
    mean: Float[Array, "state_dim"]
    cov: Float[Array, "state_dim state_dim"]

@chex.dataclass
class ParamsLGSSMDynamics:
    weights: Float[Array, "state_dim state_dim"]
    bias: Float[Array, "state_dim"]
    input_weights: Float[Array, "state_dim input_dim"]
    cov: Float[Array, "state_dim state_dim"]

@chex.dataclass
class ParamsLGSSMEmissions:
    weights: Float[Array, "emission_dim state_dim"]
    bias: Float[Array, "emission_dim"]
    input_weights: Float[Array, "emission_dim input_dim"]
    cov: Float[Array, "emission_dim emission_dim"]

@chex.dataclass
class ParamsLGSSM:
    initial: ParamsLGSSMInitial
    dynamics: ParamsLGSSMDynamics
    emissions: ParamsLGSSMEmissions

Looks good @gileshd! I'm going to merge this into main and then update it with the Parameter refactor I've been working on.


Migrate to Twitter API v2

Move every component using v1.1 to v2, if possible.

I just pushed the changes needed for migrating to Twitter API v2:
9137c2a
We still fall back to v1.1 to download video attachments though 😓 (is not supported on v2 yet).


Object matcher suggestion: toBeInstanceOf

It would be nice to be able to write:
expect(someObject).toBeInstanceOf(String.class);

It is added (among other things) in this PR:
https://github.com/mscharhag/oleaster/pull/35

Merged.
Thanks, for the merge request,


Add Dockerfile

I'm sure this is on the planning somewhere, but it would be nice to have a Docker version sooner than later.
That way, non-Java devs (like those on the CSS team) can get started fast.

See https://hub.docker.com/repository/docker/solidconformancetestbeta/conformance-test-harness


Remove instructions for import wizard.

Import wizard was deprecated, but is removed in 10.6.
Removing whole section rather than labelling with version information.

R&D team agrees to this approach.


[helper function generation] doesn't work when constructor name is different than type name

Not fixed properly.


Extending Somatic Interactions to Pathway interaction

Hi
Is it possible to get same thing like Somatic Interactions for Oncogenic Signalling Pathways ? I mean co-occurrence or mutually exclusiveness for pathways instead of genes within or among maf files
Thank you

I am closing this issue since it wont be possible.

Sorry @PoisonAlien
Please look at this figure

Figure 4 part b this article
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6420087/