Adapters do not understand Gzipped response

Description
Just as preface we are using Jest and setupPollyJest for our testing environment. It seems that @pollyjs/adapter-node-http is not returning the .json() promise of a response that's gzipped.
When we do a node-fetch call without polly
import fetch from 'node-fetch';
...
const response = await fetch('https://...', { method: 'post', headers: { 'Content-Type': 'application/json', ... }});

when we console.log a gzipped or non-gzipped response
console.log(await response.json());

we get the json object back as expected.
When we integrate polly + @pollyjs/adapter-node-http, the console.log for a gzipped response is undefined, but the plain-text (non-gzipped) response will work as expected.
Does anyone know what might be causing the gzipped response from being undefined?
Config
import NodeHttpAdapter from '@pollyjs/adapter-node-http';

export const setupPollyInTestFile = () => {
  Polly.register(NodeHttpAdapter);

  let context = setupPolly({
    adapters: [ NodeHttpAdapter ],
    adapterOptions: {
      fetch: {
        context: global
      }
    },
    }
  });

  global.pollyContext = context;
}

Dependencies
{
   "@pollyjs/core": "^2.4.0",
   "@pollyjs/persister-fs": "^2.1.0",
   "@pollyjs/adapter-node-http": "^2.4.0",
   "node-fetch": "^2.5.0",
   "setup-polly-jest": "^0.5.2",
}

Environment
Node.js v10.9.0
npm v6.2.0
darwin v18.5.0
yarn v1.13.0

@NikolasMsomething can you create a reproduction for me to debug with? That would really help in figuring out what the underlying issue is.

Thank you so much for the fast response @offirgolan! I will get a reproduction to you in the coming couple of days. Cheers.

Hello @offirgolan ! Here is a repository you can fork that almost identically mimics what dependencies I have in my development environment. If you run yarn test:refresh it will blow up any past recordings and make subsequent calls for the gzipped response. Any code regarding the actual API call lies in src/ApiUtil.js while any polly/test related code can be found in tests/jestSetup.js or tests/TestUtil.js. Thank you very much!

@NikolasMsomething thanks for creating the reproduction. I ran the test suite and the test you've provided passed.
 PASS  src/App.test.js
  ✓ fetches gzipped response and makes a proper polly recording (1629ms)

  console.log src/ApiUtil.js:10
    Response {
      size: 0,
      timeout: 0,
      [Symbol(Body internals)]:
       { body:
          Gunzip {
            _readableState: [ReadableState],
            readable: true,
            _events: [Object],
            _eventsCount: 7,
            _maxListeners: undefined,
            _writableState: [WritableState],
            writable: false,
            allowHalfOpen: true,
            _transformState: [Object],
            bytesWritten: 0,
            _handle: [Zlib],
            _hadError: false,
            _writeState: [Uint32Array],
            _outBuffer:
             <Buffer 00 00 00 00 00 00 00 00 4f 00 00 00 00 00 00 00 00 c4 81 02 01 00 00 00 8f 00 00 00 a0 20 fb 2e 01 6a 80 02 01 00 00 00 b0 69 84 02 01 00 00 00 08 00 ... >,
            _outOffset: 0,
            _level: -1,
            _strategy: 0,
            _chunkSize: 16384,
            _defaultFlushFlag: 2,
            _finishFlushFlag: 2,
            _nextFlush: -1,
            _info: undefined },
         disturbed: false,
         error: null },
      [Symbol(Response internals)]:
       { url:
          'https://us-central1-polly-issue-demo.cloudfunctions.net/respond',
         status: 200,
         statusText: 'OK',
         headers: Headers { [Symbol(map)]: [Object] },
         counter: 0 } }

  console.log src/ApiUtil.js:11
    Headers {
      [Symbol(map)]:
       [Object: null prototype] {
         'access-control-allow-origin': [ '*' ],
         'content-type': [ 'application/json; charset=utf-8' ],
         etag: [ 'W/"d-wOuS9ky6sDi1SeQD0i4Mbl1QXyY"' ],
         'function-execution-id': [ 'x3asbys3unyt' ],
         vary: [ 'Origin' ],
         'x-powered-by': [ 'Express' ],
         'x-cloud-trace-context': [ '11839f1f06c30f647025968b39a19564;o=1' ],
         'content-encoding': [ 'gzip' ],
         date: [ 'Mon, 13 May 2019 16:15:14 GMT' ],
         server: [ 'Google Frontend' ],
         'cache-control': [ 'private' ],
         'alt-svc': [ 'quic=":443"; ma=2592000; v="46,44,43,39"' ],
         connection: [ 'close' ],
         'transfer-encoding': [ 'chunked' ] } }

  console.log src/ApiUtil.js:60
    DATA IS HERE { test: true }

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.919s
Ran all test suites matching /App.test/i.
✨  Done in 3.56s.

~/src/temp/test-environment-for-netflix master* 4s
❯ node --version
v10.15.3

The issue could be with the node version you're on since we only test maintained LTS versions. I'll try running the test on v10.9.0 and see if I can reproduce it there.

Hello again @offirgolan . I made a very slight mistake when reproducing the problem , but i changed something back that now I think may definitely reproduce it for you. I just pushed the changes up now. What is different is I am now capturing the response.json() found in src/ApiUtil in a variable and trying to log that instead. Thanks for the help and sorry about that!

@NikolasMsomething any update on this issue and/or the reproduction?

@NikolasMsomething Im gonna go ahead and close this issue for now. Feel free to reopen if you're able to produce a working reproduction.


Data/Assets are missing

🐛 Bug Report
🤷♀ What did you do
Tool successfully executed (nothing changed in the configuration) in a existing strapi project, panel also starts correctly, new admin user created and then logged in -> There I see my created collections but the data including the assets are missing.
⛔ Error log
//
👩💻 Environment

📦 Node version: v16.19.1
💻 OS: Ubuntu 22.04.2 LTS

What is the content of your .dockerignore file

.tmp/
.cache/
.git/
build/
node_modules/
.env
data/

Remove the data folder from it and rebuild it to make it part of the image :)

did not help :/
i exec into container and looked for the /opt/app/public/uploads and see all of my asset files.

Check your config and variables for what database to use ☺


fix: prevent browsers from autofill on user update email form

Closes: #3519

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


chore: test with same Node.js as agoric-sdk (supported for AVA)

@kriskowal is there a reason why the "engine" field was set to ">13.0.0"?
We should not be gratuitously incompatible.

@kriskowal this now passes everything except for the (required, but no longer used) 13.x tests.
I've set things to test 12.17.0, 12.x, 14.x.  Can we land this, and change the required tests to 12.17.0?  People are going to be on 12.x for a very long time, and it's reassuring to see that this version (and newer) works just fine.


[Branch 2.7] Fix invalid rack name cause bookie join rack failed

Cherry-pick #13683 to branch 2.7
Motivation
When user set rack name to / or empty string "", it will cause all matched bookies join target rack failed and throw the following exception
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
        at java.lang.String.substring(String.java:1841) ~[?:?]
        at org.apache.bookkeeper.net.NetworkTopologyImpl$InnerNode.getNextAncestorName(NetworkTopologyImpl.java:144) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.net.NetworkTopologyImpl$InnerNode.add(NetworkTopologyImpl.java:180) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.net.NetworkTopologyImpl.add(NetworkTopologyImpl.java:425) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy.handleBookiesThatJoined(TopologyAwareEnsemblePlacementPolicy.java:717) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.handleBookiesThatJoined(RackawareEnsemblePlacementPolicyImpl.java:80) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicy.handleBookiesThatJoined(RackawareEnsemblePlacementPolicy.java:249) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy.onClusterChanged(TopologyAwareEnsemblePlacementPolicy.java:663) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl.onClusterChanged(RackawareEnsemblePlacementPolicyImpl.java:80) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicy.onClusterChanged(RackawareEnsemblePlacementPolicy.java:92) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.BookieWatcherImpl.processWritableBookiesChanged(BookieWatcherImpl.java:197) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.client.BookieWatcherImpl.lambda$initialBlockingBookieRead$1(BookieWatcherImpl.java:233) ~[io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.discover.ZKRegistrationClient$WatchTask.accept(ZKRegistrationClient.java:147) [io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at org.apache.bookkeeper.discover.ZKRegistrationClient$WatchTask.accept(ZKRegistrationClient.java:70) [io.streamnative-bookkeeper-server-<IP_ADDRESS>.jar:<IP_ADDRESS>]
        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) [?:?]
        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) [?:?]
        at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) [?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.72.Final.jar:4.1.72.Final]
        at java.lang.Thread.run(Thread.java:829) [?:?]

Which will lead ledger create failed and throw ManagedLedgerException: Not enough non-faulty bookies available
Modification

Add rack name checker on bookie rack info set.
For the has been set rack name, we add a checker in BookieRackAffinityMapping, and return null to fallback to /default-rack or /default-region/default-rack
I will add a checker in BookKeeper client side to avoid StringIndexOutOfBoundsException

Documentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?

[x] doc-required
(If you need help on updating docs, create a doc issue)

[ ] no-need-doc
(Please explain why)

[ ] doc
(If this PR contains doc changes)

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks


💡[Feature]: A complete end to end RAG chatbot application using Gemini

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Develop a Q & A RAG (Retrieval Augmented generation) chatbot on PDF document extraction using Langchain , sentence transformers and FAISS vector indexing with Gemini API that is capable of handling multiple pdf files and answering users queries using information present in pdf files. The model is deployed on Streamlit.
Workflow:

Document Parsing: Extract text from the PDF and divide it into chunks (paragraphs or sections).
Vector Embedding: Convert the chunks into vector embeddings using a pre-trained sentence transformer.
Indexing with FAISS: Store the vectorized chunks in a FAISS index for fast similarity searches.
User Query: Convert the user query into a vector and use FAISS to retrieve the most relevant text chunks from the document.
Answer Generation: Feed the retrieved chunks and the query into the Gemini API (language model) to generate a coherent answer.
Response: The chatbot returns the generated answer to the user.

Use Case

Document Analysis: Answering questions based on large reports, research papers, or legal documents.
Customer Support: Assisting customers by retrieving relevant sections from manuals, FAQs, or documentation.
Educational Tools: Providing students with answers based on textbook content or study materials.

Benefits
Efficient Retrieval: FAISS enables rapid retrieval of relevant information from large documents, making the chatbot responsive even with complex or long queries.
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

https://github.com/recodehive/machine-learning-repos/issues/1367


feat: temporary context menu

Summary
Temporary context menu for doing related-xyz actions. This will be replaced at some point by drag-and-drop interactions
Testing

Subsumed by another task.


Edges action parameter missing ?

so conventional mermaid  edges has action parameter .
for.e.g:-
graph TB
n1[new]-->|usertype submit |n2[awaiting usertype2]

in above case usertype submit is action
now in JSON canvas to mermaid how above would translate
since there is no way to add action to edge
Kindly provide clarification and guidance on the same
	edges: [{ id: 'edge1', fromNode: 'n1', toNode: 'n2',**'actiontext':'usertype submit '** }],
so is there any mechanism to add actiontext or anything like that .

Hi @alexwiench  ,
so while going through src files , I have identified there is field called "label" which can be used to add action to edges
so it would great if you add that to the readme documentation
so it looks like below
edges: [{ id: 'edge1', fromNode: 'node1', toNode: 'node2' ,label:'enter action text here'}],

Hey @matt212, thanks for looking into this!
Glad you found the label property in the JSON Canvas spec. You're right on the money - that's exactly what you need for adding action text to edges.
Just to clarify, this library is for converting existing JSON Canvas to Mermaid, not for building JSON Canvas itself. It is already set up to convert the label property to Mermaid syntax, so you're good to go!
Since I'm working with the standard JSON Canvas format, I don't typically document its specifics. But if you think it'd be helpful, I could add a link in the README to the official JSON Canvas docs.
If you are looking for an app to easily create JSON Canvas files, check out ObsidianMD. Their canvas allows you to create labels/action items by double clicking on the connecting line.

Thanks for your interest in the project. Let me know if you have any other questions!


So pieces of functionality are interchangeable, it should be possible to use different components for parts of the binding process

This library really achieves a number of independent things. These need to be decided upon. Once that is done, the architecture should be altered so that each of those responsibilities is separated in some manner, in a way in which these components may be interchangeable. These components will need to be organised somehow, such as with a peer dependency architecture.
That way, it will be possible to swap out parts of the functionality with equivalent modules.

Closely relates to #15, maybe even a dupe.

Here are the different responsibilities which the library will require

Creation of ViewModel

This will depend upon rxjs, no matter what

Getting the bindings for a given element
Getting the model context for a given binding
Loading web components
Queueing DOM updates
Walking the DOM to find elements to bind

A peer dependency architecture isn't quite the way to do this.  What I am going to do is to leave this as-is with most things in the core, but replaceable via options.
If devs want to install their own then they can via options.  I do want to split some aspects of this away to other modules though, and will attempt to identify candidates as I go.


test of galactic bloom release

see also https://github.com/ros2/rclc/pull/100

@pablogs9 just wanted to check, if new pull requests to galactic work at build farm for the bloom-release. Which is the case. So there was a problem at the build farm only.


Add root README

📝 Summary

Created an initial draft for our repo.
🔍 Description of Changes

Referred to general open-source practices for good visual appeal for the root README; added relevant badges.
📋 Checklist

[ ] I have read the contributor guidelines.
[ ] For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
[ ] I have added tests for the changes made.
[ ] I have run the code and verified that it works as expected.

📜 Reviewers

@Haleshot OR @metaboulie

It looks great

Thanks. Will work on READMEs for subfolders soon. Also need to modify existing notebooks to adhere to init.py published.


New institutions missing from superadmin list of institutions

I added Colgate as a new institution this morning. When I go to the list of institutions at
https://dmptool.org/super_admin/orgs (see dmp-institutions.jpg)
it's not included. But if I do a little URL hacking and go to
https://dmptool.org/org/admin/242/admin_edit (see dmp-colgate.jpg)
the edit page shows up.

Thanks Perry. I found the issue and will patch the Roadmap codebase.

this is fixed on roadmap stage. will test dmptool once changes are deployed.

deployed to dmptool-stg

working on dmptool-stg. closing issue.


Nightly of python3-gz-sim7 conflicts with python3-gz-sim8

Environment

OS Version: 22.04
Source or binary build? binary nightly build

python3-gz-sim8_7.999.999+nightly+git20230824+1r6dd09844c4ec9d340a857f97c0e06be44d282de1-1~jammy_amd64.deb
python3-gz-sim7 7.5.0-1~jammy

Description

Expected behavior: packages should be installable side-by-side
Actual behavior: conflicts

Steps to reproduce

In a 22.04 machine with nightly deb repositories in apt lists, run sudo apt update
sudo apt install gz-garden gz-harmonic

Output
...
Preparing to unpack .../84-python3-gz-sim8_7.999.999+nightly+git20230824+1r6dd09844c4ec9d340a857f97c0e06be44d282de1-1~jammy_amd64.deb ...
Unpacking python3-gz-sim8 (7.999.999+nightly+git20230824+1r6dd09844c4ec9d340a857f97c0e06be44d282de1-1~jammy) ...
dpkg: error processing archive /tmp/apt-dpkg-install-jo8moC/84-python3-gz-sim8_7.999.999+nightly+git20230824+1r6dd09844c4ec9d340a857f97c0e06be44d282de1-1~jammy_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/python3/dist-packages/gz/common.cpython-310-x86_64-linux-gnu.so', which is also in package python3-gz-sim7 7.5.0-1~jammy
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Selecting previously unselected package gz-harmonic.
Preparing to unpack .../85-gz-harmonic_0.0.0+nightly+git20230824+1rf013d63bea1e3dfd2d6a47d254984b3074c3495a-1~jammy_amd64.deb ...
Unpacking gz-harmonic (0.0.0+nightly+git20230824+1rf013d63bea1e3dfd2d6a47d254984b3074c3495a-1~jammy) ...
Errors were encountered while processing:
 /tmp/apt-dpkg-install-jo8moC/84-python3-gz-sim8_7.999.999+nightly+git20230824+1r6dd09844c4ec9d340a857f97c0e06be44d282de1-1~jammy_amd64.deb

Fixed by https://github.com/gazebosim/gz-sim/pull/2093


Remove trailing whitespaces and improve UPGRADE.md

Q
A

Bug fix?
No

New feature?
No

BC breaks?
No

Deprecations?
No

Fixed tickets
#915

Just making sure that there are not trailing whitespaces. Also added configuration to travis in order to check for that.
Wasn't sure about creating another PR for #915.

Ok, thanks!


Add explanatory comment about custom claims

Fixes #345

@dgrijalva super simple clarification that should allow #345 to be closed


Add configuration management

Ticket
https://mitlibraries.atlassian.net/browse/ENGX-205
Developer
Secrets

[ ] All new secrets have been added to Pantheon tiers
[ ] Relevant secrets have been updated in Github Actions
[ ] All new secrets documented in README

Documentation

[ ] Project documentation has been updated
[ ] No documentation changes are needed

Accessibility

[ ] ANDI or Wave has been run in accordance to
our guide and
all issues introduced by these changes have been resolved or opened as new
issues (link to those issues in the Pull Request details above)

Stakeholder approval

[ ] Stakeholder approval has been confirmed
[ ] Stakeholder approval is not needed

Dependencies
YES dependencies are updated
Code Reviewer

[ ] The commit message is clear and follows our guidelines
(not just this pull request message)
[ ] The changes have been verified
[ ] The documentation has been updated or is unnecessary
[ ] New dependencies are appropriate or there were no changes

I'm marking this PR as a draft - after talking with Jeremy this afternoon, we're not sure whether this is work that's going to be needed. It isn't clear yet, so I don't want to delete the PR, but I have cleaned it up a bit to focus on the relevant changes.
Still needed here are documentation for how this would be used in practice. Some of that is in the Jira ticket.

@matt-bernhardt do you mind if we close this PR? We can keep the branch around if it's still helpful and the PR itself will still be available when closed as long as it's not deleted.

I somehow missed your comment on this, Jeremy - yeah, closing this. The branch can live on, as you note.


Make balance_hash less predictable

balance_hash's are meant to hide concrete transferred amounts from MonitoringServices, but in the current form, they have not enough entropy so that an observer can easily recover the contents of balance hashes from monitoring requests.
This issue keeps track of adding a salt to balance_hash.

A native implementation causes
data/source/raiden/TokenNetwork.sol:735:9: Compiler error: Stack too deep, try removing local variables.


dubbo beans init error

Which Component
spring cloud nacos config and dubbo
Describe what problem you have encountered
I use dubbo configured mixed by xml and external yml config in nacos config server, spring failed to start with error:No registry config found or it's not a valid config! The registry config is: <dubbo:registry zookeeperProtocol="false" valid="false" prefix="dubbo.registry" />
Describe what information you have read
In spring boot , there is a way to fix  https://github.com/alibaba/nacos/issues/1597
but how to make sure nacos config is configured before dubbo bean init

I noticed that this is becasue OverrideDubboConfigApplicationListener is started earlier than NacosPropertySourceLocator  which load properties from nacos config server.
Developer can implements SpringApplicationRunListener, which done same thing as OverrideDubboConfigApplicationListener (load sys properties to dubbo properties),that works for me;
@Slf4j
public class DubboConigServerListener implements SpringApplicationRunListener, Ordered {
private final SpringApplication application;
private final String[] args;

public DubboConigServerListener(SpringApplication application, String[] args) {
    this.application = application;
    this.args = args;
}

@Override
public void contextPrepared(ConfigurableApplicationContext context) {
    SortedMap<String, Object> dubboProperties = new TreeMap<>();

    Map<String, Object> properties = EnvironmentUtils.extractProperties(context.getEnvironment());

    for (Map.Entry<String, Object> entry : properties.entrySet()) {
        String propertyName = entry.getKey();

        if (propertyName.startsWith(DubboUtils.DUBBO_PREFIX + DubboUtils.PROPERTY_NAME_SEPARATOR)
                && entry.getValue() != null) {
            dubboProperties.put(propertyName, entry.getValue().toString());
        }

    }
    log.info("dubbo properties :{}",dubboProperties);
    ConfigUtils.getProperties().putAll(dubboProperties);

}

@Override
public int getOrder() {
    return 0;
}

}


"failed to launch"

My training run crashes with the following
wandb.run_manager.LaunchError: W&B process failed to launch, see: wandb/debug.log

This is on a fresh p3.16xlarge instance, using "Deep Learning AMI (Ubuntu 18.04) Version 26.0" and pytorch_p36 environment.
Some logs below. Is there some way to have this failure be a warning and not crash my main Python process?
create web directory ./checkpoints/spade_cityscapes_dev/web...
wandb: Tracking run with wandb version 0.8.19
wandb: ERROR W&B process (PID 4087) did not respond
wandb: ERROR W&B process failed to launch, see: wandb/debug.log
Traceback (most recent call last):
  File "train.py", line 38, in <module>
    name='base',
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/wandb/__init__.py", line 1067, in init
    _init_headless(run)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/wandb/__init__.py", line 296, in _init_headless
    "W&B process failed to launch, see: {}".format(path))

debug.log
2020-01-04 02:02:26,592 DEBUG   MainThread:4087 [wandb_config.py:_load_defaults():110] no defaults not found in config-defaults.yaml
2020-01-04 02:02:26,604 DEBUG   MainThread:4087 [cmd.py:execute():728] Popen(['git', 'cat-file', '--batch-check'], cwd=/home/ubuntu, universal_newlines=False, shell=None, istream=<valid stream>)
2020-01-04 02:02:26,611 DEBUG   MainThread:4087 [cmd.py:execute():728] Popen(['git', 'rev-parse', '--show-toplevel'], cwd=/home/ubuntu, universal_newlines=False, shell=None, istream=None)
2020-01-04 02:02:26,615 DEBUG   MainThread:4087 [cmd.py:execute():728] Popen(['git', 'status', '--porcelain', '--untracked-files'], cwd=/home/ubuntu, universal_newlines=False, shell=None, istream=None)

Somehow this seems connected to directory I'm launching the script from, in one directory wandb.init() works, but when running from another directory it crashes. Deleting ./wandb didn't make difference.

I think it’s related to git.  Can you run git status from each directory and see if one hangs for a while?

Aha, that's it.
After deleting .git directory, it works as expected. "git status" doesn't hang, but it does take longer than usual (5 seconds)

Yeah, we already handle some hanging git commands, but we're clearly missing something.  If you can send me a zip of the folder with the .git directory in it that causes it we can ensure we just handle this case.  If the code is sensitive, we can probably reproduce manually.

I deleted the folder, however I imagine it could've been in a bad state -- I rsync .git folder from local machine where I do the code editing to remote machine where I do the running, so this could've been a one-off error of .git folder being corrupted

Got another failing issue like this, will send .git in email

Couldn't find work email, sent it to vanpelt gmail

Thanks<EMAIL_ADDRESS>works for next time.

Another instance I saw recently.
(pytorch_p36) yaroslavvb@sixteen:~/SPADE$ python train.py --name spade_cityscapes --dataset_mode cityscapes --dataroot ~/lab/datasets/cityscapes --batchSize 16 --gpu_ids=0,1,2,3,4,5,6,7
wandb: Tracking run with wandb version 0.8.19
wandb: ERROR W&B process (PID 19758) did not respond
wandb: ERROR Failed to kill wandb process, PID 19758
wandb: ERROR W&B process failed to launch, see: /tmp/wandb-debug.log
Traceback (most recent call last):
  File "train.py", line 15, in <module>
    sync_tensorboard=True,
  File "/home/yaroslavvb/.conda/envs/pytorch_p36/lib/python3.7/site-packages/wandb/__init__.py", line 1067, in init
    _init_headless(run)
  File "/home/yaroslavvb/.conda/envs/pytorch_p36/lib/python3.7/site-packages/wandb/__init__.py", line 296, in _init_headless
    "W&B process failed to launch, see: {}".format(path))
wandb.run_manager.LaunchError: W&B process failed to launch, see: /tmp/wandb-debug.log
(pytorch_p36) yaroslavvb@sixteen:~/SPADE$ cat /tmp/wandb-debug.log
cat: /tmp/wandb-debug.log: No such file or directory

Tried deleting wandb, .git and ../.git, still fails. Tried 15 minutes later, it works again.

In 0.8.25 we have added some protections that should prevent this.  The most common case is that the git repo either has many untracked files (usually when the training set is in the same location), or there is other .git corruption.
Currently these protections are for non-windows and non-jupyter environments.  Closing this bug by duping it to a catchall issue regarding launch errors.  If you still see this problem and do not think it is related to the duplicated bug, please reopen.  Otherwise please add new reports to:
https://github.com/wandb/client/issues/839

Duplicate of #839


Update Changelog by release operation

Automated changes by create-pull-request GitHub action

:tada: This PR is included in version 1.7.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Events

The events that are used by the Direct Line Speech might want to be separately documented.
I needed to better understand the timing and purpose of the events so I can construct a robust multi-turn exchange.
I didn't expect that I'd find the answers I was looking for in a quickstart's code comments.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 51a67123-c21c-b7c0-4390-8e439acf9fb3
Version Independent ID: 1553754f-2335-dffb-fad8-6ad8c71cac77
Content: Quickstart: Custom voice assistant, C# (UWP) - Speech service - Azure Cognitive Services
Content Source: articles/cognitive-services/Speech-Service/quickstart-voice-assistant-csharp-uwp.md
Service: cognitive-services
Sub-service: speech-service
GitHub Login: @IEvangelist
Microsoft Alias: dapine

@JonathanPalmerGD Thanks for the feedback. We are investigating the issue and will update you shortly.

Hi @JonathanPalmerGD,
Thank you for providing feedback on the docs, this is much appreciated. What exactly would you like to see and where? You mentioned like the code comments, which ones specifically? Which areas of the docs should be updated?
Thanks again, I look forward to hearing back from you.

Hi @JonathanPalmerGD,
I hope all is well. Please help me address your concerns, I had some questions about your feedback - and I'd love for you to clarify a few things. Looking forward to hearing back from you.

Howdy, sorry, this is my home account which I don't log into as much.
I view it would be helpful to have a concepts page that digs into what events occur and when.
I only learned a specific detail, that if I maintained a local variable that observed the session start/stop, I could avoid double listening. I only learned that after I re-downloaded a sample confused about the events. I can easily imagine other developers making this mistake. Especially if they are working on some existing code base.
I am actually meeting with Darren Cohen to talk about diagramming out interactions of the DL Speech to better inform our developers.

Hi @JonathanPalmerGD,
Perfect, this is very helpful -- thank you. We're going to close this issue as we're tracking the effort internally to address this feedback. Thanks again.
#please-close


Object default arguments are BAD

Occurrences of this throughout your code include:

https://github.com/trustedmercury/hiven.py/blob/90935b60899b18f6ff46df552575d8990ed0800f/hiven/gateway/http.py#L12
https://github.com/trustedmercury/hiven.py/blob/90935b60899b18f6ff46df552575d8990ed0800f/hiven/client.py#L47

This is an issue because that object will persist between calls to that function/method. Example:

What's a better solution?

Make the default be None, and check for that. Checking with is None may even be faster than using .get() on that default empty dictionary in some cases.


Microsoft Learn - Compile and run the Razor Pages failure

This is autogenerated. Please review and update as needed.
Describe the bug
Following the tutorial on Microsoft Learn. I'm using the sandbox at "Compile and run the Razor Pages" and the webapp is failing.
Command Name
az webapp up
Errors:
'NoneType' object has no attribute 'upper'
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 206, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 603, in execute
    raise ex
cli/core/commands/__init__.py, ln 661, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 652, in _run_job
    cmd_copy.exception_handler(ex)
...
cli/command_modules/appservice/custom.py, ln 2993, in webapp_up
    create_app_service_plan(cmd, rg_name, plan, _is_linux, False, sku, 1 if _is_linux else None, location)
cli/command_modules/appservice/custom.py, ln 1390, in create_app_service_plan
    sku = _normalize_sku(sku)
cli/command_modules/appservice/utils.py, ln 20, in _normalize_sku
    sku = sku.upper()
AttributeError: 'NoneType' object has no attribute 'upper'

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az webapp up

Expected Behavior
Environment Summary
Linux-4.15.0-1061-azure-x86_64-with-debian-stretch-sid
Python 3.6.5
Shell: bash

azure-cli 2.0.76

Additional Context

Please use version 2.0.77


Add Name

However I recommend you to add descriptions to your pull requests, they help people understand what your pr does 🙂


Jest tests

Anyone got Jest working?

Personally didn't do anything significantly with tape yet, but I think you would at least need to avoid stubbing of the mobservable and mobservable-react packages to test anything useful?

I don't use Tape, but rather Jest, and I do have those packages called out not to not mock them automatically but I can't get a single test to run.  Not sure what the problem is at this time.

Do you have a shareable branch somewhere? I can take a look

Just updated

npm install
npm test

https://github.com/bartonhammond/mobservable-react-native-counter

I saw that some other teams in our company have Jest / React / Mobservable tests, so it is possible at least :)

I've had nothing but trouble keeping my Jest tests working in my other
project - seems every release of RN breaks something.
Hopefully you find a team that can provide some insight
Thanks for looking at it.
On Tue, Feb 16, 2016 at 8:48 AM, Michel Weststrate<EMAIL_ADDRESS>
wrote:

I saw that some other teams in our company have Jest / React / Mobservable
tests, so it is possible at least :)
—
Reply to this email directly or view it on GitHub
https://github.com/mweststrate/mobservable-react/issues/28#issuecomment-184710070
.

Wow, Jest is a real horror to setup, I changed the setup a bit that got me at least beyond the initial "FAIL" I got without any further error when including components.
See: https://github.com/mweststrate/mobservable-react-native-counter/commit/76dba47625d0062e4f7fc652e1a4ba26ee48596f
The script preprocesser seems to fix it that at least the tests start when referring to a component. But after that it throws compile errors. Note sure how to setup multiple preprocessors. It seemed that the require('react-native') import in the mobservable-react module broke without the preprocess. But so far Jest is one big mistery for me. I happily continue not using it :wink:

Rats!  Sorry to hear you didn't make any headway!  I had all my fingers and toes crossed.
The ReactJS Conf is going on now - I've been watching it.  They haven't mentioned Jest.  No surprise -it's the read headed step child.
Once I get my server stuff settled down I want to pick up the mobservable rewrite w/ my Snowflake/Redux app.  It would be nice to run tests.  Maybe by then I'll have some insights.

@mweststrate is there any chance that compatibility with Jest will be fixed soon?

@kitze  From your tweets it seems the question has been answered, but for future reference
jest.mock('react-dom'); seems to be needed (https://github.com/facebook/react/issues/7386). See also https://semaphoreci.com/community/tutorials/how-to-test-react-and-mobx-with-jest


Spaces filters

Description
This adds two possible filter terms (driveType, id) to the /me/drives endpoint on the graph api. These can be used with the odata query parameter "$filter".
We only support the "eq" operator for now.
This is leveraging the spaces-registry branch and needs to be rebased after that one has been merged.
Examples
Filter by Id
curl -L -X GET 'https://localhost:9200/graph/v1.0/me/drives?$filter=id eq '\''f98b7cc7-15ba-4715-9d16-23a746dfe648'\''' \
-H 'Authorization: Basic YWRtaW46YWRtaW4='

Filter by type
curl -L -X GET 'https://localhost:9200/graph/v1.0/me/drives?$filter=driveType eq '\''personal'\''' \
-H 'Authorization: Basic YWRtaW46YWRtaW4='

Motivation and Context

How Has This Been Tested?

test environment:
test case 1:
test case 2:
...

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Technical debt
[ ] Tests only (no source changes)

Checklist:

[x] Code changes
[ ] Unit tests added
[ ] Acceptance tests added
[ ] Documentation ticket raised:

@butonic @kobergj I think I found a bug.
Regardless of the filters, the shared jail (driveType=="virtual") is always present.

:boom: Acceptance test localApiTests-apiArchiver-ocis failed. Further test are cancelled...

@kobergj let me add some API tests


[IOTDB-251]improve TSQueryDataSet structure in RPC

The original definition of TSQueryDataSet leads to the row-at-a-time writing process listed as follows:
- TSIService.fetchResults_result.write
  - TSFetchResultsResp.write
    - TSQueryDataSet.write
      - TSRowRecord.write
        - TSDataValue.write
          - TBinaryProtocol.writeDouble
            ---------------
            - TIOStreamTransport.write (这里实际是TSocket.write) 阻塞式socker
              ---------------
              - BufferedOutputStream.write

Therefore, by defining a more compact TSQueryDataSet, the time cost of TSIService.fetchResults_result.write may likely be decreased.
The original definition of TSQueryDataSet:
struct TSFetchResultsResp{
	1: required TSStatus status
	2: required bool hasResultSet
	3: optional TSQueryDataSet queryDataSet
}

struct TSQueryDataSet{
	1: required list\<TSRowRecord> records
}

struct TSRowRecord{
  1: required i64 timestamp
  // column values
  2: required list\<TSDataValue> values
}

struct TSDataValue{
  1: required bool is_empty
  2: optional bool bool_val
  3: optional i32 int_val
  4: optional i64 long_val
  5: optional double float_val
  6: optional double double_val
  7: optional binary binary_val
  8: optional string type
}

The new TSQueryDataSet in this pr:
struct TSQueryDataSet{
   1: required binary values
   2: required i32 rowCount
}

Test is on the way.

Credit to @samperson1997 , in his experiment, this pr can decrease the time cost of fetchResults_result.write to 27% of its original value.

How about this:
Deprecate the List of RowRecord and instead use a RowBatch, which avoids too many objects such as Fields and RowRecord itself.

Hi, thank you, I have recorded your thought on JIRA https://github.com/apache/incubator-iotdb/pull/455 for future work.

Thanks for your work. The compact view of dataset will definitely improve the efficiency of network transport. I'm ready to have a try!


Remove ?extractAssets workaround in favor of build.ssrEmitAssets

Implements https://github.com/vikejs/vike/issues/1339.

Is this the right direction?

I think it's easier if we inject the server manifest into the client manifest here. That way, the rest of Vike's code will work as it does today, e.g. no need to modify node/runtime/globalContext.ts then. Atually, if we keep dist/assets.json semantically equivalent, then I think we don't need to modify the server runtime at all.

I'm reading ssrEmitAssets.ts right now, and I'm seeing logic to merge manifests. So I guess you already tried to merge the server manifest into dist/assets.json and have things "just work".
Still, I don't think we need to adapt the server runtime (nor the pre-rendering code) if we keep dist/assets.json semantically equiavelent. (More precisely speaking, all we need to do is to remove the ?extractAssets handling from the server runtime, which you seem to have already done.)

AFAICT, if we do a clever merging instead of just a "dumb"  { ...clientManifestParsed, ...serverManifestParsed }, then we can keep assets.json semantically equivalent and, therefore, alleviate ourselfes the need to adapt the server runtime.
So I guess the question is this: can we make the merging so clever that evertyhing else just works as before? A lot of Vite frameworks use ssrEmitAssets; a GitHub search for "ssrEmitAssets" would point to a lot of code to copy from.

LGTM, let me know if for you as well then I'll merge.
Looking forward to further resolve all blockers and merge the worker server PR ✨

Looks good, works for me.


Fabric Fluid API

Resolves #17

TODO: Fluid items (ie. buckets)

There's a bit of an annoying bug where fluids flicker to 0 when being inserted... Not sure exactly why.


How can F* assert result matching with array

I try to use assert with Quicksort function in tutorial but got error:
Error SortPoly.fst(66,0-66,48): An unknown assertion in the term at this location was not provable
How can i do in this case ?
module QuickSort
val mem: 'a -> list 'a -> Tot bool
let rec mem a l = match l with
  | [] -> false
  | hd::tl -> hd=a || mem a tl
val append : list 'a -> list 'a -> Tot (list 'a)
let rec append l1 l2 = match l1 with
  | [] -> l2
  | hd :: tl -> hd :: append tl l2
val append_mem:  l1:list 'a
    -> l2:list 'a
    -> Lemma (requires True)
        (ensures (forall a. mem a (append l1 l2) = (mem a l1 || mem a l2)))
        [SMTPat (append l1 l2)]
let rec append_mem l1 l2 = match l1 with
  | [] -> ()
  | hd::tl -> append_mem tl l2
val length: list 'a -> Tot nat
let rec length l = match l with
  | [] -> 0
  | _ :: tl -> 1 + length tl
val sorted: list int -> Tot bool
let rec sorted l = match l with
    | [] -> true
    | [x] -> true
    | x::y::xs -> x <= y && sorted (y::xs)
val partition: ('a -> Tot bool) -> list 'a -> Tot (list 'a * list 'a)
let rec partition f = function
  | [] -> [], []
  | hd::tl ->
     let l1, l2 = partition f tl in
     if f hd
     then hd::l1, l2
     else l1, hd::l2
val partition_lemma: f:('a -> Tot bool)
    -> l:list 'a
    -> Lemma (requires True)
    (ensures ((length (fst (partition f l)) + length (snd (partition f l)) = length l
        /\ (forall x. mem x (fst (partition f l)) ==> f x)
        /\ (forall x. mem x (snd (partition f l)) ==> not (f x))
        /\ (forall x. mem x l = (mem x (fst (partition f l)) || mem x (snd (partition f l)))))))
    [SMTPat (partition f l)]
let rec partition_lemma f l = match l with
    | [] -> ()
    | hd::tl -> partition_lemma f tl
val sorted_concat_lemma: l1:list int{sorted l1}
    -> l2:list int{sorted l2}
    -> pivot:int
    -> Lemma (requires ((forall y. mem y l1 ==> not (pivot <= y))
        /\ (forall y. mem y l2 ==> pivot <= y)))
        (ensures (sorted (append l1 (pivot::l2))))
        [SMTPat (sorted (append l1 (pivot::l2)))]
let rec sorted_concat_lemma l1 l2 pivot = match l1 with
    | [] -> ()
    | hd::tl -> sorted_concat_lemma tl l2 pivot
let cmp i j = i <= j
val sort: l:list int -> Tot (m:list int{sorted m /\ (forall i. mem i l = mem i m)})
                            (decreases (length l))
let rec sort l = match l with
  | [] -> []
  | pivot::tl ->
    let hi, lo = partition (cmp pivot) tl in
    append (sort lo) (pivot::sort hi)

let test2 = assert((sort [3;5;7;6;8;9]) = [3;5;6;7;8;9])

F* is not really meant for doing full evaluation of complex expressions at verification time. First, this won't be efficient. Second, there is a fixed bound up to which F* is unfolding fixpoints. If you don't care about efficiency you can set this bound much higher using these flags:
  --initial_fuel non-negative integer  Number of unrolling of recursive functions to try initially (default 2)
  --max_fuel non-negative integer  Number of unrolling of recursive functions to try at most (default 8)
  --min_fuel non-negative integer  Minimum number of unrolling of recursive functions to try (default 1)

If you care about efficiency then just extract your code to OCaml using --codegen OCaml and then test it in OCaml.

If you go for the first option you might also want to set the --z3timeout flag to something large ... and then go get a coffee or something :)

@catalin-hritcu: How can i ensure that this is correct with my below bubble sort code:
module BubbleSort
val length: list 'a -> Tot nat
let rec length l = match l with
| [] -> 0
| _ :: tl -> 1 + length tl
val sorted: list int -> Tot bool
let rec sorted l = match l with
| [] -> true
| [x] -> true
| x::y::xs -> x <= y && sorted (y::xs)
let rec swap lst =
match lst with
|[] -> []
|x::[] -> x::[]
|x::y::[] -> if x > y then y::x::[]
else x::y::[]
|x::y::xs -> if x > y then
y::(swap (x::xs))
else
x::(swap (y::xs))
let rec sort x count = if count > 1 then sort (swap x) (count - 1)
else x
let bubleSort x = sorted (sort x (length x))
let testBubleSort x = sort x (length x)

Sorry, but I don't understand your question.

@catalin-hritcu :
I want to make sure let sort x (length x) work correctly with  sorted (sort x (length x)) = true;
How can i write this.
Thank you

This is not something a tool can prove automatically for you. You can have a look here:
https://github.com/FStarLang/FStar/blob/master/examples/algorithms/insertionsort.fst
at how a similar theorem can be manually proved.

@catalin-hritcu:
One more question. So i can not try this code:
val bubleSort : l:list int -> Tot (m:list int{sorted m})
let rec bubleSort l = match l with
| [] -> []
| x::xs -> bubleSort (sort xs (length xs))
Because i got BubbleSort.fst(29,0-31,34) : Error
Computed type "m:(list int){(sorted m)}" and effect "ALL" is not compatible with the annotated type "m:(list int){(sorted m)}" effect "Tot"
Can you help to explain more about this.
Thank you

Dear @hungvq00390,
I think the code for of the bubbleSort function is simply:
let bubleSort x = sort x (length x)

You would like to prove two things.
Totality: that is quite easy to prove by annotating the type of the functions with "decreases" clauses":
val swap : lst:list int -> Tot (list int) (decreases (length lst))
let rec swap lst =
    match lst with
    |[] -> []
    |x::[] -> x::[]
    |x::y::xs -> if x > y then
                    y::(swap (x::xs))
                else
                    x::(swap (y::xs))

val sort : list int -> count:nat -> Tot (list int) (decreases count)
let rec sort x count = if count > 1 then sort (swap x) (count - 1)
                       else x

val bubleSort : list int -> Tot (list int)
let bubleSort x = sort x (length x)

(I also simplified the code for swap since one computation was redundant.)
Functional correctness: in order to prove that the result of bubbleSort is indeed sorted, you will need to state good invariants for the functions swap and sort... which might not be that easy.
Hope this helps,
Chantal.

Everything here seems answered, closing.


Don't pass empty model to QCompleter

It's empty by default.

Tested on Debian Linux with no issues observed.


GitBook Community in Google+

Hi DomainDrivenArchitecture,
I am Aleen (https://github.com/aleen42) , and the founder of GitBook Community in Google+: https://plus.google.com/communities/110696788983472771007, which aims to sharing books or plugins of GitBook, talking anything about it as well. Here I am sincerely inviting you to join us.  :punch:

joined :-)

@jerger well done~


Release settings?

Have you considered adding some documentation on using Swifter in shipping products? I just attempted to build a project including Swifter for an Ad Hoc build and it failed. I'd prefer not to change the project settings for the framework, but am unclear if that will be necessary to get the project to build ad hoc or not.

Same here. :(

Outdated discussion.


Updates to trigger behaviour table

@swylezol,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@swylezol : Thanks for your contribution to the Azure documentation! The author, @MandiOhlinger, has been notified to review your proposed change.

#sign-off


python311Packages.json-tricks: 3.17.2 -> 3.17.3

Diff: https://github.com/mverleg/pyjson_tricks/compare/refs/tags/v3.17.2...v3.17.3
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 251804 run on x86_64-linux 1

4 packages built:

python310Packages.json-tricks
python310Packages.json-tricks.dist
python311Packages.json-tricks
python311Packages.json-tricks.dist


[DebuggingTheCompiler] Document a few flags for dumping llvm-ir.

I needed to use these and realized they were not documented here. I had them in
my brain, but others may not, so seemed good to do.

@swift-ci smoke test and merge

@swift-ci smoke test and merge

@swift-ci smoke test linux platform

@swift-ci smoke test linux platform

@swift-ci smoke test linux platform


Meta: Write README and rename repo?

The README.md is currently quite empty, it would be nice to add a proper description of the game, screenshot/gif and link to the itch.io page.
It might also be worth renaming the GitHub repository (the previous name will stick work as a redirection) to better reflect what it is about? (e.g. daemon-vs-demon or gameoff-2017)

thanks for taking the initiative on this!


Unable to use custom certs with OpenAI client

I'm unable to get custom cert bundle path (client side) working with OpenAI. For background, I'm on a work laptop which is behind a VPN, so we use a custom SSL cert (.pem file) for any outbound requests. The Python requests library seems to honor this with the use of the REQUESTS_CA_BUNDLE env variable, but was unable to get it working with the OpenAI client.
Here's what I tried so far:
import os

import openai

print(os.environ['REQUESTS_CA_BUNDLE'])  # confirm that it prints out correct path to .pem file

openai.verify_ssl_certs = False # disable verify but this does nothing, only logs a warning
openai.api_key = 'my-api-key'

file = '/path/to/my/file'
with open(file, 'rb') as in_file:
    openai.File.create(
        file=in_file, purpose='search')

This results in the below error being printed to console:
/Users/rnag/path/to/pem/file
/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py:47: UserWarning: verify_ssl_certs is ignored; openai always verifies.
  warnings.warn("verify_ssl_certs is ignored; openai always verifies.")
Traceback (most recent call last):
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connection.py", line 416, in connect
    self.sock = ssl_wrap_socket(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/files (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py", line 255, in request_raw
    result = _thread_context.session.request(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/files (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/path/to/py/file", line 11, in <module>
    openai.File.create(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_resources/file.py", line 43, in create
    response, _, api_key = requestor.request("post", url, files=files)
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py", line 98, in request
    result = self.request_raw(
  File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py", line 266, in request_raw
    raise error.APIConnectionError("Error communicating with OpenAI") from e
openai.error.APIConnectionError: Error communicating with OpenAI

My workaround
Its not ideal but current workaround I'm using is to manually update the api_requestor.py script to pass in verify=False to request method:
timeout=TIMEOUT_SECS,
verify=False,

Hi @rnag!  Thanks for filing this issue.
As a general security rule we'd very much prefer to verify all requests which is why we don't allow setting verify=False.
It should be possible to connect to our API with just a curl command (https://beta.openai.com/docs/api-reference/making-requests) or just a standalone requests.post.  Can you try either of those with your certs and see if the issue persists?

I'm going to close this for now.  Feel free to reopen!

So after further testing, it looks like I was running into the same issue with requests library, but not with curl. I realized that for some reason the issue was that I was exporting REQUESTS_CA_BUNDLE in my env to point to the path of my client cert file. For some reason requests and openai client in python didn't like that. Once I unset this variable, everything started magically working, and the cert errors disappeared on my end. So I would agree that this issue should (hopefully) be resolved now.


custom type definitions

Up until now, type checking has relied on primitive types. For example, if the argument is an {object}, it's safe to infer that it is very likely a Vega Lite specification, but this based on a human stylistic preference which is not actually enforced by the type system. For the most part this works due to a combination of discipline and functional programming style which rewards the use of primitive arguments that don't need custom types at all.
This decision was mostly an aesthetic objection to the tedium of needing to /import() the @typedef every time it is referenced in every single function header, which is the standard JSDoc syntax, but it's also possible to import once using a standard ES import.

The obvious remaining gap is that the Vega Lite specification object is still typed as simply an object instead of something more sophisticated.


Stations can not reasonably be refilled after minor-major atmos loss.

Description
This recent PR: #28909 Has made it so if the pressure drops then vents go in to a lockout mode to stop air refilling after a  spacing and not wasting air from distro.
But to have them allow to push gas out again you are required to on the air alarm change the mode to fill.
When there is a mass venting situation It is not a reasonable expectation that all of these alarms would have someone go to and do the fill option.
Reproduction

Vent station
See pressure lockout.
Manually fill from airlock.

Screenshots

Additional context
Seen a couple of times now, Doubt if education or additional access to these air sensors it would be able to solve it. Might need to have some form of mass action or trigger where the lockout would come off?

Might not be much of a problem after observing it for another day, will probably close this soon. Seems like its only issue when mass sections have low pressure.

Me when I assign the tags before I scroll all the way down.

You mentioned wrong PR, that PR added the override to the UnderPressureLockout(UPLO) so people could just set Fill mode instead of having to run around opening doors and/or bringing canisters to raise the pressure.
The UnderPressureLockout that you're speaking of was added in #9824 ,
The reason this has become a problem was because the UPLO was set to 80kPa in #29504 , which is a temporary fix to get rid of the situation where this lockout doesn't get triggered at all.
Currently I have a PR which should get rid of this problem, after that gets merged the UPLO will be set back to some reasonable level, so opening doors has a good chance of repressurizing rooms.


FileSystem Tab: Inline renaming of files and folders

The current SceneTree allows for direct inline edit of node names while the FileSystem docker does not. In fact, it still uses a popup text dialog. The pull request is a simple proposal to enable inline editing in the FileSystem docker while maintaining feature parity.
Effectively, it replaces the first image by the second.

The addition of the void set_editor_selection(int start, int end); is maybe not the definite best solution, however there are no other means of accessing the LineEdit in thew Tree class (other than making it public or adding a getter). Additionally, I'm not sure why TreeItems have the editable property set to "not".
This is my first pull request for Godot so please let me know if there's anything I missed. This seemed like a rather simple change.
Bugsquad edit: Closes https://github.com/godotengine/godot-proposals/issues/4933

Thanks for your contribution. You need to squash the commits as required by our pipeline (see https://docs.godotengine.org/en/latest/community/contributing/pr_workflow.html).

Just squashed the commits as required.

Fixed some style errors.

Moved the set_editable in the _create_tree() and _update_tree(). For some reasons, the favorites creation is done within the _update_tree().
That way, the Favourite: and "res://" can't be edited.

Wrong names are not reverted and display incorrectly:

I fixed the wrong names not being reverted. I considered two options:

Simply calling update_tree() which is the simple way; or
Restoring old_name from to_rename with some string magic which needs to handle file vs folders

i.e. String old_name = to_rename.path.is_file ? to_rename.path.get_file() : to_rename.path.substr(0, to_rename.path.length() - 1).get_file();
And simply go with selected_item->set_text(old_name); // approximation
My understanding is that update_tree() will be eventually called on a successful rename so it shouldn't be a performance problem to call is for a rename call.
I also noticed that accidentally renaming files such as ".tscn" or ".svg" seemed an undesired behavior.  I added a guard in the error checks since the file would be lost for the editor and hidden in the filesystem.
I haven't touched the rest of the rename_operation method since I frankly have no idea what the rest of thecode does.

Restoring old_name from to_rename with some string magic which needs to handle file vs folders

I think this would be preferred. There are no performance considerations here, but if the tree update is not necessary it should be avoided. Although the current solution is ok too, of improving it turns out too difficult for whatever reason. You could use my suggestion above.

I also noticed that accidentally renaming files such as ".tscn" or ".svg" seemed an undesired behavior. I added a guard in the error checks since the file would be lost for the editor and hidden in the filesystem.

Not sure what do you mean by this.

If a file named "scene.tscn" is renamed to ".tscn" by accident or voluntarily, the rename will pass because the file length check includes the extension. On my linux machine, this also means that the file is hidden from godot as its now starting with a dot.
I made the changes, I used the suggestion as it makes sense.
However, I have to figure whether my changes have done anything because double clicks currently edit the item instead of opening them.

The code in tree.cpp at lines 3563 is causing some weird behaviors.
Specifically, if the cell is set as "editable", a double click will automatically enable the "inline edit" mode and not send a double click signal.
if (rect.has_point(mpos)) {
	if (!edit_selected()) {
		emit_signal(SNAME("item_double_clicked"));
	}
} else {
	emit_signal(SNAME("item_double_clicked"));
}

Option 1. Optional argument to edit_selected() to bypass the selectable check and do the inline edit anyway.
Option 2. Add a flag to Tree class to allow for selection of default double click behavior. (probably the best one)

I fixed the wrong names not being reverted.

Seems to be still happening. Tested on folders with duplicate names.

Alright, I have made changes, but also a huge mistake with git I believe.

I created a new pull request instead
https://github.com/godotengine/godot/pull/76794


Hide unused modules from listings menu in head.php

Adds a conf item called conf('show_unused_modules') with default setting of TRUE.
When set to FALSE, the Listings menu will show only modules that are active in conf['modules'].

As a sidenote, @poundbangbash suggested hiding unused modules based on whether there are items in the database table associated with the module.
If that is preferred, I could try implementing that instead.

I love this feature but am not in love with the implementation.

I believe this feature would be much more clear if you changed the variable to hide_nonactive_modules.
2nd, the default value in config_default.php should be false to match current MR implementation
Lastly the wording after the variable change would be

When true, modules that are not in conf['modules'] will not be shown in the 'Listings' drop down menu.

Obviously, these are all my own opinions. If you disagree or @bochoven wants changes go with his opinion
Overall 👍

Oh my lord yes this is excellent. I think looking to conf['modules'] is appropriate - there may be modules that I tested with a few machines and the database has some tables and data, that I do not end up deploying widely.


consider using String instead of int to reprsent Contents.Card.Number

this is not yet the case, but it could happen that the number of a card contains letters or other symbols
as a precaution, and consistency with the collector number type elsewhere, I'd suggest to change the number type from int to string
            "card": [
              {
                "foil": false,
                "name": "Thorn Elemental",
                "number": 11,
                "set": "o90p",
                "uuid": "6a4a698e-42d7-54ec-97b1-aa7cc54f2bd5"
              }
            ],

Card numbers could be either int or string depending on need. They've been unified as strings.


Remove slice_io APIs

Tracking issue to either document, find a valid pattern that can't be solved with posix based IO, or remove the slice_io APIs. As suggested by @flaub, it might make the most sense to just remove these.
The only example that uses these APIs is the waldo example
https://github.com/risc0/risc0/blob/d2502d46270a1fe0f0b53059ce8de1fc662539c7/examples/waldo/src/bin/prove.rs#L121
and it seems like a reasonable use case, so probably best to start there to make sure that pattern can be handled well without it before migrating away from the slice_io APIs (if that decision is made).
Links to some of the APIs:
https://github.com/risc0/risc0/blob/d2502d46270a1fe0f0b53059ce8de1fc662539c7/risc0/zkvm/src/host/client/env.rs#L306
https://github.com/risc0/risc0/blob/d2502d46270a1fe0f0b53059ce8de1fc662539c7/risc0/zkvm/src/host/client/env.rs#L315
https://github.com/risc0/risc0/blob/d2502d46270a1fe0f0b53059ce8de1fc662539c7/risc0/zkvm/src/guest/env.rs#L298
https://github.com/risc0/risc0/blob/d2502d46270a1fe0f0b53059ce8de1fc662539c7/risc0/zkvm/src/host/client/slice_io.rs#L28

Since this isn't a blocking issue, I'm going to close for now. Perhaps we can reopen in the future, but I suspect the API will change a bunch so this becomes obsolete.


hasSumStats returning incorrect data in API

Still not sure whether it can be an API issue or a data issue, but hasSumStats is returning false instead of true for the next study GCST005921.
GraphQL API query
Related to #378
Reported by @Jeremy37 and @mirandaio

Resolved


Normal right click context menu option

Normal right click context menu option with Copy & Paste menu items like in a normal Windows context, etc. which replaces the current right click direct actions.

Description of the new feature/enhancement

Proposed technical implementation details (optional)

Thanks for the suggestion! Please make sure to search for duplicates when filing new issues. This is already being tracked by another issue on our repo - please refer to #3337 for more discussion.
/dup #3337


Looking for AVAssetWriterInputPixelBufferAdapter support

I'm trying to use this crate in my Rust app (thanks to @morajabi for the suggestion!)
Was looking for this class: https://developer.apple.com/documentation/avfoundation/avassetwriterinputpixelbufferadaptor
But I suppose it doesn't exist yet. How would one add it? I'm happy to spend some time creating a PR if any of the maintainers don't have time for it but can point me in the right direction.

Sorry for delay, was moving to different apartment this weekend.
If you need another other missing api's feel free to request it or use this commit as ref.

No worries man! Congrats on the new apartment and thanks for the fab work!


Warning/Maintenance notification in dashboard

Related https://github.com/CartoDB/cartodb-platform/issues/5505
We need to show the users a warning/notification block that we are going to set in the notification field.
We used to have something like that in the old dashboard, I'd love to go back and make a screenshot but I wasn't able, to announce maintenance tasks for example.
So the basic idea is to insert a text in the notification field for a user or group of users using a rake. Then, when that user enters in her dashboard that message appears in the front page to warn the user.
Adding @inigomedina to the issue as PM, @elenatorro as the current RT person for awareness and probably we have to add someone from design.
Take into account that this should be an easy and quick thing that could even be disposable.

@inigomedina we need to take action on this to unblock one of the tasks for the migration of pg11

From the frontend side, we'll need this new user field to know if there's any notification. We already have a parameter for organization notifications, but this would be different. We need to know where should we show these notifications from a design perspective, but we could do a PoC to create the needed components in the new dashboard and then apply the design.

we'll need this new user field to know if there's any notification

We're providing it. Is a field called notification inside the user_data. We can rename it but is available

ok! then I can start working on it 👍 Adding the rt-frontend label to this task

Cool thank you @elenatorro !

I created the component to start working on it (https://github.com/CartoDB/cartodb/pull/14866), but we need help from the @CartoDB/design-team to implement it.

Thanks, people, for including me here.
@ana-md is shadowing me in this initiative and a nice first exercise for her is to participate at this specific action. Could you explain to her how she should work on this?
Thanks!

I think the first thing to do is work with someone from the design team and get final message we're going to show to the users to communicate this kind of warnings/notifications. It's important because looks like we're going to use this kind of message to notify about important things like maintenance, migrations and so on but as the PM you have the whole picture :)

Thanks @ethervoid and @inigomedina for adding me here! Is this message the one we are just showing to the FREE users? Or also the one for the Enterprise? As I understood yesterday, the downtime will be longer for the FREE users and for the Enterprise we will schedule 10 minutes in a specific day and time, isn't it? Do we want to mention that at all in the warning?

From my point of view, this issue is about adding a general notification warning for any kind of warning message we want to show at any time. Therefore, we need a common design for these notifications, so we can make use of them when necessary.
About the specific message for this case (thanks @ana-md for mentioning it!) this warning accepts html content, which means you can customize it, for example, with bold texts.

Thanks @elenatorro for the explanation!
Adding design team in this discussion to decide how we want to show this warnings to the users in general from a design perspective. After talking to @alonsogarciapablo it seems that there wasn't any specific idea/plan for this purpose when the dashboard was firstly designed. @kukukaka what do you think about this?

Hi!
In fact this kind of big / really noticeable notification in the Dashboard could be useful also for the pricing-related warnings: eg. "you are getting close to reaching the maximum number of datasets in your plan" or "you have reached the XXX limit, some features might be disabled...".
Pinging @alejandraarri to be aware of this conversation

Hi @VictorVelarde!
How can I see that notification?
It's also useful for #1579
Thank you!

@VictorVelarde, don't worry about it!
@ana-md just explained it to me :)

@newLadyAga Yes!, my comment was on the path of a possible reuse of the component, and I'm glad if it is that way.
It that makes sense from Design, then the next step would be to bring the backenders & frontenders in. But that's your next move! 😄

@VictorVelarde we're in vector, we're watching you...

It'd be awesome to reuse the component for other purposes, +1 to this. However, I thought that we needed to have a notification urgently (that's why it's on RT) in order to notify the users about the PG11 migration as explained here. So, in my opinion:

If we think we can design and implement a reusable component for multiple purposes before the PG11 migration, then go for it!

If this is going to take longer than what it'd be to design something simpler for a unique component, then I'd try to focus on this specific task and after that create something better from what we have to fit other use cases.

@elenatorro I completely agree on that. First the original objective, specially if it is urgent. Then, the oportunity to reuse

Yes, @elenatorro's approach is the right one: next week we are going to start with the first migrations so this week we need to start notifying that.
@kukukaka, do you think we could have a first iteration to be used this week?

Exactly :) after talking with infra, we would need to have this on Thursday, but remember that this Wednesday is holiday for most of us.

Hi!
I'm working on it, but just to clarify: all notifications should go into the notification page /dashboard/notifications. That page was mean to grow to be the notification hub (even if at the moment we only had org notifications and the ones we send). That's where the limit notifications etc should be displayed (+the limits module).
So I understand is just a wording issue, but I guess this would be more a warning, or a more highlighted notification (right now we show them with a green dot in the user avatar).

Hi again,
I'm sharing with you 2 options to highlight the notification.
The first one is already built and is the same mechanism we use to highlight the Feedback from inside the user dropdown. As is a pattern we already use, I would prefer to use it.
The second one is more visible and is a common pattern. But it would be a new component.
https://invis.io/WDRZCTM6Z2Q
Cheers.

That's awesome ✨ ! Let me try to add the notification to the dropdown then. Thank you very much 👍

Awesome, @kukukaka! 👏
The current notification system expects by default 3 types of message? Long, medium, short? Or are we able to launch just with one of them?

It looks really cool @kukukaka !
For the pg11 upgrade we will show the short notification with the following message:

To improve our platform, there is an upcoming scheduled maintenance on DD-MM-YYY from XX:XX to YY:YY. The normal operation of the platform is not guaranteed and any new data added during that time could potentially be lost.

@elenatorro please let me know when we have it implemented to start testing it in staging :)

Here you can see the notification working on staging. From my PoV we should make the text bigger but there you can see text with HTML tags () and it works great :)

I randomly bumped into this in staging when testing something else (I just happened to chose the testwarning account 😝):

Also:

⚠ For some reason the header has disappeared!

Another comment / question... not sure if we're using the user's local timezone, but just in case, should we include the timezone to make it more explicit?

Another comment / question... not sure if we're using the user's local timezone, but just in case, should we include the timezone to make it more explicit?

Probably yes, makes sense to make it clear to everyone and avoid confusion. Thanks for pointing it out @alonsogarciapablo

I've fixed the issue we had in the "old" dashboard pages (profile, account, api-keys...). I currently have a problem with the 'close' button in the notification. This button would require more implementation as it is right now in the old pages to be aligned with the new ones. I think the best solution, for now, and since this is for a specific use case, remove the close button and create a separate task to improve the integration of these notifications with the new dashboard and with the user profile pages.
What do you think?

Yes, @elenatorro. Your approach sounds absolutely reasonable and pragmatic. Let's do it! 😃

Deployed 🚀 thanks everyone for the hard work 💪

VAMOS @elenatorro! 👏👏👏


Mark plugin as compatible with v3 of serverless

It's needed so users can upgrade to v3 of the Framework without issues

I think it would be better to do this in https://github.com/getlift/lift/pull/131 and merge it.

@t-richard good point. If #131 will be merged on time, then this can safely be closed.
Still, otherwise, this plugin will work without issues in v3m as is in master, so just taking this in and publishing it before v3 is released, will ensure that users have no issues in upgrading when v3 is released

Thanks! Finally merged #131


Fix change log contributors manually for 0.61.0 against superseded PR #9816

As title. The contributor of #9816 should go into the change log for 0.61.0, reasons are given in this comment https://github.com/numba/numba/pull/9816#issuecomment-2532486081. This will need a manual fix.

This is done in #9843


Hashed shouldn't provide AsMut

Currently, Hashed provides AsMut, which allows mutable access to the wrapped value. This means that the value can change, without the pre-computed hash also being changed. Either AsMut should be removed, or self.hash should be replaced with Option<u64> (or Option<NonZeroU64>, and the internal logic updated so that mutable access clears the hash so it can be recomputed next time hash is called.

You're completely right - I'll put up a patch to remove the AsMut. Making it an Option would be possible, but given we hash on construction, it feels like a weird corner case to occasionally lose the hash.

Done in 6327eea9d087f375c71832c30fdfc9902a4e4b3a


Query API pagination MUST be detected by clients

Currently, Query API pagination "SHOULD" be supported by implementations but MUST be detected by clients.
There's nothing in the RAML specification of the Query API to indicate that pagination is not required (see definition of the paged trait. The APIs - Query Parameters - Pagination documentation does begin "Query APIs SHOULD support pagination..." however.
From the client's point of view this means that a simple Query API request to e.g. /x-nmos/query/v1.2/nodes (with no query parameters) can have two different meanings.

For a Query API which doesn't implement pagination, all nodes are returned in a single response
For a Query API which does implement pagination, only the first page of results is returned

This problem arises because pagination isn't 'opt in' from the client's point of view. This is quite different than the other optional API features, where the client has to make a different request to opt into the feature, e.g. query.downgrade, query.rql, query.ancestry_id.
In the case of pagination, it is currently therefore critical that the client distinguish between these two cases by examining the response headers that are (I believe) required if pagination is implemented, e.g. X-Paging-Limit.
Suggestions:

Clarify in the RAML that the Query API traits are all currently optional (SHOULD or MAY)
Clarify in the RAML, and the Query Parameters documentation, that clients currently MUST check the pagination response headers to distinguish a paginated response
Add more tests for pagination in https://github.com/AMWA-TV/nmos-testing
Think about how to automate tests for clients
Think about whether to 'fix' this in some way in IS-04 v1.3+

@garethsb-sony has this been adequately addressed via the PR and your own work in the test suite, or do you think something further is required?

The test suite doesn't check clients at all.
So unfortunately, I think the PR words are the best we have right now.
I do wonder whether pagination should have been opt-in for clients.
I.e.
/nodes?foo=bar with no paging parameters must return all matching resources, which may be unordered, but a response of (probably) 501 is permitted if that operation would be too intensive.
/nodes?foo=bar&paging.limit=100 would opt into the 'paging' trait specification and return the first 100 matching resources (or fewer, including if the server's maximum limit is less), ordered as specified.

Ok. As an opt-in mechanism would be a new feature for v1.3+ I'll add a new issue to document the thinking at least, then it's not forgotten about and can be re-visited if desired.


Old app version in released 0.8.4

In  rbac-manager 0.8.4 i found that application version still 0.8.3 in "version/version.go"

And also here https://github.com/FairwindsOps/rbac-manager/blob/0.8.4/deploy/all.yaml#L161


MirrorStrategy is not working with tf.estimator.DNNClassifier

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
Changed from models/samples/core/get_started/premade_estimator.py
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Ubuntu 16.64
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
No
TensorFlow installed from (source or binary):
Compile from source 1.10
TensorFlow version (use command below):
1.10
Python version:
2.7.12
Bazel version (if compiling from source):
0.16.1
GCC/Compiler version (if compiling from source):
5.4.0
CUDA/cuDNN version:
9.1
GPU model and memory:
2 x TITAN Xp/12GB
Exact command to reproduce:
python premade_estimator.py

Describe the problem
I try to use MirrorStrategy to parallel premade_estimator.py. The  code is below. I made two changes:
(1) I make a fake input_fn to build (features, labels) from random values.
(2) I add two lines to use MirrorStrategy when initializing the conf when building estimator.
It woks fine if I do not use MirrosStrategy(without (2)), but fails as following logs output.
BTW: I also tried the original input_fn, reported the same errors.
Appreciate the help
Source code / logs
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import argparse
import tensorflow as tf

import iris_data

parser = argparse.ArgumentParser()
parser.add_argument('--batch_size', default=32, type=int, help='batch size')
parser.add_argument('--train_steps', default=1000, type=int,
                    help='number of training steps')

CSV_COLUMN_NAMES = ['SepalLength', 'SepalWidth',
                    'PetalLength', 'PetalWidth', 'Species']

def train_input_fn_fake(features_name, batch_size):
    features_len = len(features_name)
    data_len = 10000
    features = tf.random_uniform([data_len, features_len], minval=0, maxval=1, dtype=tf.float32)
    labels = tf.random_uniform([data_len, 1], minval=0, maxval=2, dtype=tf.int32)
    def map_dnn_input(features, labels):
        features = tf.split(features, features_len)
        features = dict(zip(features_name, features))
        return features, labels
    dataset = tf.data.Dataset.from_tensor_slices((features, labels)).map(map_dnn_input)
    dataset = dataset.shuffle(1000).repeat().batch(batch_size)
    return dataset

def main(argv):
    args = parser.parse_args(argv[1:])

    # Feature columns describe how to use the input.
    my_feature_columns = []
    my_features = []
    for key in CSV_COLUMN_NAMES:
        my_features.append(key)
        my_feature_columns.append(tf.feature_column.numeric_column(key=key))

    # configure multi gpu
    distribution = tf.contrib.distribute.MirroredStrategy(num_gpus=2)
    config = tf.estimator.RunConfig(train_distribute=distribution)

    # Build 2 hidden layer DNN with 10, 10 units respectively.
    classifier = tf.estimator.DNNClassifier(
        feature_columns=my_feature_columns,
        # Two hidden layers of 10 nodes each
        hidden_units=[100, 100],
        # The model must choose between 3 classes.
        n_classes=3,
        # Config
        config = config
        )

    # Train the Model.
    classifier.train(
        input_fn=lambda:train_input_fn_fake(my_features,
                                            args.batch_size),
        steps=args.train_steps,
        )

if __name__ == '__main__':
    tf.logging.set_verbosity(tf.logging.INFO)
    tf.app.run(main)

Log output:

2018-08-21 12:27:58.645341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:01:00.0
totalMemory: 11.91GiB freeMemory: 10.62GiB
2018-08-21 12:27:58.874232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Found device 1 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:02:00.0
totalMemory: 11.91GiB freeMemory: 11.74GiB
2018-08-21 12:27:58.875049: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1485] Adding visible gpu devices: 0, 1
2018-08-21 12:27:59.303516: I tensorflow/core/common_runtime/gpu/gpu_device.cc:966] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-21 12:27:59.303548: I tensorflow/core/common_runtime/gpu/gpu_device.cc:972]      0 1
2018-08-21 12:27:59.303553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:985] 0:   N Y
2018-08-21 12:27:59.303557: I tensorflow/core/common_runtime/gpu/gpu_device.cc:985] 1:   Y N
2018-08-21 12:27:59.303931: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1098] Created TensorFlow device (/device:GPU:0 with 10261 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-08-21 12:27:59.405388: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1098] Created TensorFlow device (/device:GPU:1 with 11361 MB memory) -> physical GPU (device: 1, name: TITAN Xp, pci bus id: 0000:02:00.0, compute capability: 6.1)
INFO:tensorflow:Device is available but not used by distribute strategy: /device:CPU:0
INFO:tensorflow:Configured nccl all-reduce.
2018-08-21 12:27:59.555447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1485] Adding visible gpu devices: 0, 1
2018-08-21 12:27:59.555515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:966] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-21 12:27:59.555524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:972]      0 1
2018-08-21 12:27:59.555530: I tensorflow/core/common_runtime/gpu/gpu_device.cc:985] 0:   N Y
2018-08-21 12:27:59.555535: I tensorflow/core/common_runtime/gpu/gpu_device.cc:985] 1:   Y N
2018-08-21 12:27:59.555693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1098] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10261 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-08-21 12:27:59.555813: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1098] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 11361 MB memory) -> physical GPU (device: 1, name: TITAN Xp, pci bus id: 0000:02:00.0, compute capability: 6.1)
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:Partitioned variables are disabled when using DistributionStrategy.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:batch_all_reduce invoked for batches size = 6 with algorithm = nccl, num_packs = 1, agg_small_grads_max_bytes = 0 and agg_small_grads_max_group = 10
INFO:tensorflow:Error reported to Coordinator:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 297, in stop_on_exception
    yield
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/mirrored_strategy.py", line 175, in _call_for_each_tower
    **merge_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 660, in _distributed_apply
    self._create_slots(var_list)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/adagrad.py", line 73, in _create_slots
    with ops.colocate_with(v):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 4080, in _colocate_with_for_gradient
    with self.colocate_with(op, ignore_existing):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 4132, in colocate_with
    op = internal_convert_to_tensor_or_indexed_slices(op, as_ref=True).op
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1285, in internal_convert_to_tensor_or_indexed_slices
    value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1124, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/values.py", line 445, in _tensor_conversion_mirrored
    assert not as_ref
AssertionError
Traceback (most recent call last):
  File "premade_estimator.py", line 84, in <module>
    tf.app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "premade_estimator.py", line 78, in main
    steps=args.train_steps,
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 343, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 1166, in _train_model
    return self._train_model_distributed(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 1294, in _train_model_distributed
    self.config)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/distribute.py", line 857, in call_for_each_tower
    return self._call_for_each_tower(fn, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/mirrored_strategy.py", line 418, in _call_for_each_tower
    return _call_for_each_tower(self, fn, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/mirrored_strategy.py", line 181, in _call_for_each_tower
    coord.join(threads)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join
    six.reraise(*self._exc_info_to_raise)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 297, in stop_on_exception
    yield
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/mirrored_strategy.py", line 175, in _call_for_each_tower
    **merge_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 660, in _distributed_apply
    self._create_slots(var_list)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/adagrad.py", line 73, in _create_slots
    with ops.colocate_with(v):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 4080, in _colocate_with_for_gradient
    with self.colocate_with(op, ignore_existing):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 4132, in colocate_with
    op = internal_convert_to_tensor_or_indexed_slices(op, as_ref=True).op
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1285, in internal_convert_to_tensor_or_indexed_slices
    value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1124, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/values.py", line 445, in _tensor_conversion_mirrored
    assert not as_ref
AssertionError

cc @ispirmustafa the authority on this issue.

I got the same errors when training using MirroredStrategy.

I got the same error when training using MirroredStrategy with AdamOptimizer and AdagradOptimizer

@yncxcw The issue with AdagradOptimizer should be fixed now. Can you try running your model again with the latest TensorFlow code? Thanks!
@olel-may I am not able to reproduce the issue with the AdamOptimizer. Do you have a standalone snippet of code which reproduces this issue?

Why I still get this error? Do I need to renew the Tensorflow?

@AmberCheng What version of TensorFlow are you using? Can you try with the latest version? If you still get this error, can you post a snippet of code that reproduces the issue?

@AmberCheng What version of TensorFlow are you using? Can you try with the latest version? If you still get this error, can you post a snippet of code that reproduces the issue?

Yep, I have install the Tensorflow 1.11, and there is no more this problem. Thank you~~~

This issue should be fixed now. Please reopen this if you still encounter this error. Thanks!


[Python] How to build a python 3.8 wheel?

I am trying to build a stripped down python 3.8 wheel from tag apache-arrow-3.0.0. I am building it using
docker-compose run -e PYTHON=3.8 -e PYTHON_VERSION=3.8 python-wheel-manylinux-2014
however, it is being built using python 3.6.
The final wheel gets tagged with pyarrow-3.0.0-cp36-cp36m-manylinux2014_x86_64.whl and looking through the build log I can see that cmake decides to use python 3.6 even though python 3.8 is available in the manylinux image.
cmake -DARROW_BROTLI_USE_SHARED=OFF -DARROW_BUILD_SHARED=ON -DARROW_BUILD_STATIC=OFF -DARROW_BUILD_TESTS=OFF -DARROW_DATASET=ON -DARROW_DEPENDENCY_SOURCE=SYSTEM -DARROW_DEPENDENCY_USE_SHARED=OFF -DARROW_FLIGHT==OFF -DARROW_GANDIVA=OFF -DARROW_HDFS=OFF -DARROW_JEMALLOC=ON -DARROW_MIMALLOC=ON -DARROW_ORC=OFF -DARROW_PACKAGE_KIND=manylinux2014 -DARROW_PARQUET=ON -DARROW_PLASMA=OFF -DARROW_PYTHON=ON -DARROW_RPATH_ORIGIN=ON -DARROW_S3=OFF -DARROW_TENSORFLOW=OFF -DARROW_USE_CCACHE=ON -DARROW_UTF8PROC_USE_SHARED=OFF -DARROW_WITH_BROTLI=OFF -DARROW_WITH_BZ2=OFF -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=/tmp/arrow-dist -DCMAKE_TOOLCHAIN_FILE=/opt/vcpkg/scripts/buildsystems/vcpkg.cmake -DCMAKE_UNITY_BUILD=OFF -DOPENSSL_USE_STATIC_LIBS=ON -DThrift_ROOT=/opt/vcpkg/installed/x64-linux/lib -DVCPKG_TARGET_TRIPLET=x64-linux-static-release -G Ninja /arrow/cpp
-- Building using CMake version: 3.19.2
-- The C compiler identification is GNU 9.3.1
-- The CXX compiler identification is GNU 9.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-9/root/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-9/root/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Arrow version: 3.0.0 (full: '3.0.0')
-- Arrow SO version: 300 (full: 300.0.0)
-- clang-tidy not found
-- clang-format not found
-- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
-- infer not found
-- Found Python3: /opt/python/cp36-cp36m/bin/python3.6 (found version "3.6.12") found components: Interpreter

Am I missing any config to make it build a python 3.8 wheel?
Thanks.

I had an answer on the pyArrow mailing list. Instead of passing the PYTHON environment variable as a -e option, I had to set it before the build command like so:
PYTHON=3.8 docker-compose run python-wheel-manylinux-2014


add flag to set MPI IO collective or back to default (independent)

(Thanks for sending a pull request! Please make sure you click the link above to view the contribution guidelines, then fill out the blanks below.)
What does this implement/fix? Explain your changes.
The collective flag allows to use compression with MPI IO
Does this close any currently open issues?
should close #332
Any relevant logs, error output, etc?
n.a.
Any other comments?
n.a.
Where has this been tested?
Operating System: …
Platform: …

I just reverted the change in the exampleMPI.cpp
I tested it. It is good for me.

Thanks a lot!