MSID: 97662 Version: 2 DOI: 10.1101/2<IP_ADDRESS>6563

MSID: 97662
Version: 2
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>6563
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=97662
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>6563

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/97662v2
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>6563
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/97662/v2/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

warn: [biorender-check] Article text contains a mention of bioRender. Since bioRender do not permit content to be published under a CC-BY license (https://help.biorender.com/en/articles/8601313-creative-commons-licensing-for-biorender-figures-premium-only), if any images were created using bioRender a blanket permissions statement will need to be added for the Reviewed Preprint: 'Any parts of this image created with BioRender are not made available under the same license as the Reviewed Preprint, and are © 2024, BioRender Inc.'
warn: [journal-ref-text-content] This journal reference (id c4) has untagged textual content - :eadg1485.doi:. Is it tagged correctly?
warn: [journal-ref-text-content] This journal reference (id c25) has untagged textual content - /eLife.92731. Is it tagged correctly?
warn: [journal-ref-text-content] This journal reference (id c52) has untagged textual content - :ii23-31. . Is it tagged correctly?

asking authors to confirm which figure(s) made using biorender

Hi @rkcook312, have the authors responded to this at all? If not do you think it's worth chasing?

no response, chaser sent


How to draw MAP and loss grap?

In the readme file, the WikiQA rank task gives the loss and MAP graph, but how we can get the loss and MAP information at each 10 Iters? And how to draw the graph like that you drawed?

I have the following functions to generate the results from the log file, you can try it.
def reduce_result(x, interval):
    y = {}
    count = {}
    for iter, value in x:
        idx = iter / interval
        if idx not in y:
            y[idx] = 0.0
            count[idx] = 0
        y[idx] += value
        count[idx] += 1
    for idx in y:
        y[idx] /= count[idx]

    # convert dict to list
    idx = 0
    rtn = []
    while True:
        if idx in y:
            rtn.append(y[idx])
            idx += 1
        elif idx == 0:
            rtn.append(0)
            idx += 1
        else:
            break
    return rtn

def generate_log_data(filename, red_train = 1000, red_valid = 1000, red_test = 1000):
    reduce_interval = {
        "train" : red_train,
        "valid" : red_valid,
        "test" : red_test
        }
    pattern_raw = r"\[(.+?)\]\t\[(.+?):(.+?)\] Iter:(.+?)\t(.+?)$"
    pattern_res = re.compile(r"(.+?)=(.+?)\t")
    log_lines = {}
    for line in open(filename):
        m = re.match(pattern_raw, line)
        if m:
            tag = m.group(3)
            iter = int(m.group(4))
            all_res = m.group(5) + '\t'

            if tag not in log_lines:
                log_lines[tag] = {}
            for p in pattern_res.findall(all_res):
                node = p[0]
                val = float(p[1])
                if node not in log_lines[tag]:
                    log_lines[tag][node] = []
                log_lines[tag][node].append([iter, val])
    for tag in log_lines:
        for node in log_lines[tag]:
            log_lines[tag][node] = reduce_result(log_lines[tag][node], reduce_interval[tag])
    return filename, log_lines


TypeError: Cannot read properties of undefined (reading 'type')

I don´t know if i got the right database id, but with the id i currently have in the config file every sixty seconds when the module updates itself it shows the error "TypeError: Cannot read properties of undefined (reading 'type')". If I use another database id it shows another error which says it cannot find a database with that id.

@choffmann it is the right id (i'm pretty sure about that)

@Mister1Max1 Okay, i will take a look :eyes:

@choffmann idk how but I`ve managed to get it working. But it want to know if its possible to do not show tasks which are already checked (and if the anwser is yes with which code?)

@Mister1Max1 Not at the moment. I am currently working on restructuring the project and adding new features. I will consider your idea


graphql-codegen fails to load GraphQLSchema

Describe the bug
graphql-codegen fails to load Schema when GraphQLSchema is provided via a custom loader.
However, what is more interesting is, the same configuration may sometimes work, and all seems to be go smooth. But, you will encounter this problem, as you come to it in a later time.
The error is reproducable, and I have provided a log of the error in error.log file. Please refer to the repository.
To Reproduce
Please Refer: https://gitlab.com/errors1/codegen-err
It is easily reproducable by executing npm run cgn on the given repository.

My GraphQL schema:

// Just for the purpose of providing root type Query
import gql from "graphql-tag";

export const gqlTypeDefs = gql`
  type Query{
    user: User!
  }

  type User {
    id: ID
    name: EmailAddress!
    email: String!
  }
`;

It works when you change the type EmailAddress above, and remove 'test.js' from the codegen.yml

My GraphQL operations:

N/A

My codegen.yml config file:

overwrite: true
schema:
- "test.js"
- "sample_schema.ts"
documents: null
generates:
  tsdefs.ts:
    plugins:
      - "typescript"
      - "typescript-resolvers"
  ./graphql.schema.json:
    plugins:
      - "introspection"

My Custom Loader

const {typeDefs} = require("graphql-scalars");
const {makeExecutableSchema} = require("graphql-tools");

const schema = makeExecutableSchema({typeDefs});
console.log(schema);

module.exports = schema;

Expected behavior
The GraphQLSchema should work as a Schema Type as described in the docs.
Environment:

OS: Windows 10 Ver. 1809
@graphql-codegen/...: 1.14.0
NodeJS: 10.16.0 && 12.16.3

Additional context
I tried reducing the typeDefs array returned by 'graphql-scalars' into a single string, and converting it to GraphQLSchema, it worked for a while. But, the error began to pop up again.
If you can't find the error after a while, try changing relaunching the shell, or your dev enviornment. The errors pops up again.
Solution I am using right now.
// custom loader
let {typeDefs} = require("graphql-scalars");
typeDefs = typeDefs.reduce((a, c) => a+c+' ', '');
module.exports = typeDefs;

Here, the type exported is string instead of GraphQLSchema, whereas GraphQLSchema work.

@pavittarx I tried your reproduction, and I noticed this:

The test.js is not exporting an executable schema - it just exports types. So if your config file points only to it, you'll get:

  ✖ ./graphql.schema.json
    Error: Query root type must be provided.
        at assertValidSchema (/Users/dotansimha/Dev/temp/codegen-err/node_modules/graphql/type/validate.js:71:11)
        at assertValidExecutionArguments (/Users/dotansimha/Dev/temp/codegen-err/node_modules/graphql/execution/execute.js:136:35)
        at executeImpl (/Users/dotansimha/Dev/temp/codegen-err/node_modules/graphql/execution/execute.js:84:3)
        at execute (/Users/dotansimha/Dev/temp/codegen-err/node_modules/graphql/execution/execute.js:62:35)

If you export typeDefs from sample_schema - it will work, but if you are trying to export GraphQLSchema from that file, it will fail - because it tries to compile the schema within your file, without the types from test.js so it will fail.

You configuration should include a single schema loader file, that loads the types from graphql-scalars, and builds the schema with those types and your custom types as GraphQLSchema object. If you export a GraphQLSchema object - it should contains everything and should be executable.
If you wish to expose typeDefs and let codegen merge and make your schema executable, you can do it with the same configuration you have in this reproduction.
I noticed a bug with the graphql-tools loaders, I guess we should allow telling the loaders that you prefer to use require over AST lookup, we'll try to fix that.

@dotansimha We have noPluck option for @graphql-tools/code-file-loader, so you can skip Tag Pluck and use require directly.
https://github.com/ardatan/graphql-tools/blob/master/packages/loaders/code-file/src/index.ts#L111

You are right @ardatan :) @pavittarx your custom loader file should look like that:
const { makeExecutableSchema } = require("graphql-tools");
const { typeDefs: scalarsTypeDefs } = require("graphql-scalars");
const gql = require('graphql-tag');

const gqlTypeDefs = gql`
  ${scalarsTypeDefs}

  type Query{
    user: User!
  }

  type User {
    id: ID
    name: EmailAddress!
    email: String!
  }
`;

module.exports = makeExecutableSchema({ typeDefs: gqlTypeDefs });

And the codegen config file:
schema:
  - ./sample_schema.js:
      noPluck: true
generates:
  tsdefs.ts:
    plugins:
      - "typescript"
      - "typescript-resolvers"
  ./graphql.schema.json:
    plugins:
      - "introspection"

I'll update the docs now!

@cancan101 you can use custom document loader, and use require in your custom loader to load it from your code file. This way you'll get the interpolated version of the documents and feed it to codegen.
See: https://graphql-code-generator.com/docs/getting-started/documents-field#custom-document-loader

I tried with this in yml + added a loader and don't see the custom loader being used:
schema:
  - "schema.graphql"
documents:
  - "./src/**/*.{ts,tsx}":
    loader: "my-documents-loader.js"

@cancan101 can you please try like that? (YAML is very sensitive when it comes to indentation)
schema:
  - "schema.graphql"
documents:
  - "./src/**/*.{ts,tsx}":
      loader: "my-documents-loader.js"

Ah okay, that fixed the issue with the loader however I am getting the following error using the example loader from the linked docs:
    GraphQLError: Syntax Error: Unexpected Name "import".

You need to add ts-node as well :) because NodeJS itself doesn't know how to load TS files.

Sure, that makes sense (and this is likely due to my ignorance) but after installing ts-node, how do I get the generate script to run / load using it?

Sure, that makes sense (and this is likely due to my ignorance) but after installing ts-node, how do I get the generate script to run / load using it?

You need to specify require config key, see: https://graphql-code-generator.com/docs/getting-started/require-field#typescript-support

Still not quite working. A couple questions:

Should I be able to use the example document loader linked here:? https://graphql-code-generator.com/docs/getting-started/documents-field
Does the require key need to be set under documents or globally in the yml file? I have:

overwrite: true
schema:
  - "schema.graphql"
documents:
  - "./src/**/*.{ts,tsx}":
      loader: "./my-documents-loader.js"

generates:
  src/generated/graphql.tsx:
    plugins:
      - "typescript"
      - "typescript-operations"
      - "typescript-react-apollo"
    config:
      skipTypename: true
      withHooks: false
      withHOC: false
      withComponent: false
      withMutationFn: false
      namingConvention:
        enumValues: keep
  ./graphql.schema.json:
    plugins:
      - "introspection"

require:
  - ts-node/register

Error I am getting:
      ✖ Load GraphQL documents
        → Syntax Error: Unexpected Name "import".

Could you create a reproduction on CodeSandbox so we can help you better?


Tweak search query filtering

Look at Google Cloud console for example on adding an "X" button to clear any search query.

the Chip component would be excellent for this


Adding initial events from fendermint

Adding initial set of events for logging from fendermint.
The approach is rather simple at this stage to avoid complex structs/macro. The event type is centralised in core crate then other crate can call emit! to emit the event in the format that is most suitable.

@fridrik01 The rest of the metrics are from harder to get, most are from combetbft I think. The ABCI ones we can use #[instrument], should be easily doable.


How can i enable Arcore positional Tracking on ALVR 2.3.1?

Hello.
i cant find the option to enable arcore positional tracking with ALVR 2.3.1 using my Exynos S7E with my gear vr.
anyone please tell me how can i enable arcore positional tracking?
Thank You.

+1 to this, i would love to play around with this option after trying it in quakevr but i don't see any option to enable this. please help

@BossBolognese ikr me too how do we enable this anyone please help us.

I've figured out how to make it work. you need to grab the newest client apk from https://github.com/polygraphene/ALVRClient/releases and install it manually. the oculus store version lacks the positional tracking. i used gearvr injector to sighn the apk and it worked fine, however the tracking is very inaccurate (as to be expected) and my phone gets hot very quickly. still fun to mess around with though.

@BossBolognese omg ima try it right now tysm

@BossBolognese also would recommend you to get a fan i bought my fan from aliexpress for like almost 20$ and it keeps my phone cool while using gear vr

could you maybe give me a link to that fan?

@BossBolognese my app on my exynos s7 edge keeps crashing now have u experienced that?


100% CPU utilization of the ovn-northd process

Test Environment

OVN Version：20.03

OVN_NB：Standalone Mode

OVN_SB：Standalone Mode

OVN-Northd：Standalone Mode

OVN-Controller：33 nodes

OS：CentOS Linux release 8.0.1905 (Core)

Kernel：4.18.0-80.el8.x86_64

Recently, OVN performance pressure test is being carried out, this time based on standalone mode.
After the stress test, the CPU utilization of the ovn-northd process has been very high, close to 100%,  but the CPU utilization of the OVN-NB/OVN-SB process is very low, close to 0.
1. The OVN-NB log during the stress test as follows
2020-06-29T02:01:09.667Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.12.0
2020-06-29T02:01:19.677Z|00003|memory|INFO|6656 kB peak resident set size after 10.0 seconds
2020-06-29T02:01:19.677Z|00004|memory|INFO|cells:31 monitors:2 sessions:1
2020-06-29T02:48:42.312Z|00005|memory|INFO|peak resident set size grew 56% in last 2842.6 seconds, from 6656 kB to 10352 kB
2020-06-29T02:48:42.312Z|00006|memory|INFO|cells:34572 monitors:2 sessions:2
2020-06-29T02:50:32.331Z|00007|memory|INFO|peak resident set size grew 54% in last 110.0 seconds, from 10352 kB to 15956 kB
2020-06-29T02:50:32.332Z|00008|memory|INFO|cells:139273 monitors:2 sessions:2
2020-06-29T02:53:52.381Z|00009|memory|INFO|peak resident set size grew 52% in last 200.0 seconds, from 15956 kB to 24316 kB
2020-06-29T02:53:52.381Z|00010|memory|INFO|cells:299166 monitors:2 sessions:2
2020-06-29T03:00:22.523Z|00011|memory|INFO|peak resident set size grew 52% in last 390.1 seconds, from 24316 kB to 36872 kB
2020-06-29T03:00:22.523Z|00012|memory|INFO|cells:537462 monitors:2 sessions:2
2020-06-29T03:00:32.532Z|00013|memory|INFO|peak resident set size grew 116% in last 10.0 seconds, from 36872 kB to 79768 kB
2020-06-29T03:00:32.532Z|00014|memory|INFO|cells:542471 monitors:2 sessions:2
2020-06-29T03:16:32.928Z|00015|memory|INFO|peak resident set size grew 79% in last 960.4 seconds, from 79768 kB to 142532 kB
2020-06-29T03:16:32.928Z|00016|memory|INFO|cells:918878 monitors:2 sessions:2
2020-06-29T04:03:20.635Z|00017|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.651Z|00018|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.651Z|00019|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.662Z|00020|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.707Z|00021|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.713Z|00022|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.727Z|00023|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.748Z|00024|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.748Z|00025|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.763Z|00026|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:25.019Z|00027|memory|INFO|peak resident set size grew 54% in last 2812.1 seconds, from 142532 kB to 219124 kB
2020-06-29T04:03:25.019Z|00028|memory|INFO|cells:1599256 monitors:2 sessions:2

2. The OVN-SB log during the stress test as follows
2020-06-29T02:01:09.728Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.12.0
2020-06-29T02:01:19.738Z|00003|memory|INFO|6656 kB peak resident set size after 10.0 seconds
2020-06-29T02:01:19.738Z|00004|memory|INFO|cells:227 monitors:2 sessions:1
2020-06-29T02:37:25.624Z|00005|jsonrpc|WARN|unix#2: send error: Broken pipe
2020-06-29T02:37:25.625Z|00006|reconnect|WARN|unix#2: connection dropped (Broken pipe)
2020-06-29T02:37:27.489Z|00007|jsonrpc|WARN|unix#3: send error: Broken pipe
2020-06-29T02:37:27.490Z|00008|reconnect|WARN|unix#3: connection dropped (Broken pipe)
2020-06-29T02:39:56.123Z|00009|jsonrpc|WARN|unix#6: receive error: Connection reset by peer
2020-06-29T02:39:56.123Z|00010|reconnect|WARN|unix#6: connection dropped (Connection reset by peer)
2020-06-29T02:40:46.428Z|00011|jsonrpc|WARN|unix#8: receive error: Connection reset by peer
2020-06-29T02:40:46.428Z|00012|reconnect|WARN|unix#8: connection dropped (Connection reset by peer)
2020-06-29T02:48:12.367Z|00013|memory|INFO|peak resident set size grew 90% in last 2812.6 seconds, from 6656 kB to 12628 kB
2020-06-29T02:48:12.367Z|00014|memory|INFO|cells:46047 monitors:3 sessions:34
2020-06-29T02:48:22.370Z|00015|memory|INFO|peak resident set size grew 151% in last 10.0 seconds, from 12628 kB to 31644 kB
2020-06-29T02:48:22.370Z|00016|memory|INFO|cells:97656 monitors:3 sessions:34
2020-06-29T02:49:12.608Z|00017|memory|INFO|peak resident set size grew 67% in last 50.2 seconds, from 31644 kB to 52816 kB
2020-06-29T02:49:12.608Z|00018|memory|INFO|cells:419671 monitors:3 sessions:34
2020-06-29T02:50:03.187Z|00019|memory|INFO|peak resident set size grew 62% in last 50.6 seconds, from 52816 kB to 85820 kB
2020-06-29T02:50:03.187Z|00020|memory|INFO|cells:729595 monitors:3 sessions:34
2020-06-29T02:51:14.654Z|00021|memory|INFO|peak resident set size grew 52% in last 71.5 seconds, from 85820 kB to 130108 kB
2020-06-29T02:51:14.654Z|00022|memory|INFO|cells:1116719 monitors:3 sessions:34
2020-06-29T02:53:34.956Z|00023|memory|INFO|peak resident set size grew 55% in last 140.3 seconds, from 130108 kB to 201172 kB
2020-06-29T02:53:34.957Z|00024|memory|INFO|cells:1813179 monitors:3 sessions:34
2020-06-29T02:57:27.556Z|00025|memory|INFO|peak resident set size grew 52% in last 232.6 seconds, from 201172 kB to 305564 kB
2020-06-29T02:57:27.556Z|00026|memory|INFO|cells:2789603 monitors:3 sessions:34
2020-06-29T03:04:24.792Z|00027|timeval|WARN|Unreasonably long 7210ms poll interval (6019ms user, 674ms system)
2020-06-29T03:04:24.792Z|00028|timeval|WARN|faults: 315533 minor, 0 major
2020-06-29T03:04:24.792Z|00029|timeval|WARN|disk: 0 reads, 304840 writes
2020-06-29T03:04:24.792Z|00030|timeval|WARN|context switches: 1376 voluntary, 602 involuntary
2020-06-29T03:04:24.792Z|00031|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=db894c43:
2020-06-29T03:04:24.792Z|00032|coverage|INFO|hmap_pathological          7.4/sec    10.200/sec        4.8067/sec   total: 17306
2020-06-29T03:04:24.792Z|00033|coverage|INFO|hmap_expand              42731.0/sec  8333.533/sec      757.0369/sec   total: 2726445
2020-06-29T03:04:24.792Z|00034|coverage|INFO|lockfile_lock              0.0/sec     0.000/sec        0.0000/sec   total: 1
2020-06-29T03:04:24.792Z|00035|coverage|INFO|poll_create_node         129.2/sec   218.500/sec      145.9342/sec   total: 525996
2020-06-29T03:04:24.792Z|00036|coverage|INFO|poll_zero_timeout          0.0/sec     0.000/sec        0.0261/sec   total: 99
2020-06-29T03:04:24.792Z|00037|coverage|INFO|seq_change                 0.0/sec     0.000/sec        0.0000/sec   total: 3
2020-06-29T03:04:24.792Z|00038|coverage|INFO|pstream_open               0.0/sec     0.000/sec        0.0003/sec   total: 3
2020-06-29T03:04:24.792Z|00039|coverage|INFO|util_xalloc              2314482.0/sec 445723.267/sec    41307.0894/sec   total: 148731186
2020-06-29T03:04:24.792Z|00040|coverage|INFO|71 events never hit
2020-06-29T03:04:24.792Z|00041|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.792Z|00042|memory|INFO|peak resident set size grew 405% in last 417.2 seconds, from 305564 kB to 1541964 kB
2020-06-29T03:04:24.792Z|00043|memory|INFO|cells:4077200 monitors:3 sessions:34
2020-06-29T03:04:24.793Z|00044|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.794Z|00045|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.795Z|00046|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.795Z|00047|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.796Z|00048|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.797Z|00049|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.798Z|00050|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.798Z|00051|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.799Z|00052|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:34:28.261Z|00053|timeval|WARN|Unreasonably long 13569ms poll interval (11639ms user, 1074ms system)
2020-06-29T03:34:28.262Z|00054|timeval|WARN|faults: 614892 minor, 0 major
2020-06-29T03:34:28.262Z|00055|timeval|WARN|disk: 0 reads, 573808 writes
2020-06-29T03:34:28.262Z|00056|timeval|WARN|context switches: 2355 voluntary, 1133 involuntary
2020-06-29T03:34:28.262Z|00057|coverage|INFO|Skipping details of duplicate event coverage for hash=db894c43
2020-06-29T03:34:28.262Z|00058|poll_loop|INFO|Dropped 63 log messages in last 1804 seconds (most recently, 1801 seconds ago) due to excessive rate
2020-06-29T03:34:28.262Z|00059|poll_loop|INFO|wakeup due to [POLLIN] on fd 18 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:274 (94% CPU usage)
2020-06-29T03:34:28.262Z|00060|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.262Z|00061|memory|INFO|peak resident set size grew 87% in last 1803.5 seconds, from 1541964 kB to 2888308 kB
2020-06-29T03:34:28.262Z|00062|memory|INFO|cells:7666606 monitors:3 sessions:34
2020-06-29T03:34:28.263Z|00063|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.264Z|00064|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.264Z|00065|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.265Z|00066|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.266Z|00067|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:37.858Z|00068|timeval|WARN|Unreasonably long 9593ms poll interval (9501ms user, 58ms system)
2020-06-29T03:34:37.858Z|00069|timeval|WARN|faults: 919 minor, 0 major
2020-06-29T03:34:37.858Z|00070|timeval|WARN|context switches: 0 voluntary, 10 involuntary
2020-06-29T03:34:37.858Z|00071|coverage|INFO|Skipping details of duplicate event coverage for hash=db894c43
2020-06-29T03:34:37.858Z|00072|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:37.859Z|00073|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:37.861Z|00074|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:37.862Z|00075|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:40.262Z|00076|poll_loop|INFO|Dropped 442 log messages in last 3 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-29T03:34:40.262Z|00077|poll_loop|INFO|wakeup due to [POLLOUT] on fd 17 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:153 (100% CPU usage)

3. After the stress test, the ovn-northd log as follows
2020-06-29T04:32:02.261Z|07385|timeval|WARN|faults: 4 minor, 0 major
2020-06-29T04:32:02.261Z|07386|timeval|WARN|context switches: 24 voluntary, 5 involuntary
2020-06-29T04:32:02.261Z|07387|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/ovn/ovnsb_db.sock) at lib/stream-fd.c:157 (99% CPU usage)
2020-06-29T04:32:06.519Z|07388|timeval|WARN|Unreasonably long 4259ms poll interval (4245ms user, 0ms system)
2020-06-29T04:32:06.519Z|07389|timeval|WARN|faults: 128 minor, 0 major
2020-06-29T04:32:06.519Z|07390|timeval|WARN|context switches: 16 voluntary, 1 involuntary
2020-06-29T04:32:06.520Z|07391|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (101% CPU usage)
2020-06-29T04:32:10.769Z|07392|timeval|WARN|Unreasonably long 4250ms poll interval (4236ms user, 0ms system)
2020-06-29T04:32:10.770Z|07393|timeval|WARN|faults: 128 minor, 0 major
2020-06-29T04:32:10.770Z|07394|timeval|WARN|context switches: 12 voluntary, 3 involuntary
2020-06-29T04:32:15.024Z|07395|timeval|WARN|Unreasonably long 4255ms poll interval (4238ms user, 0ms system)
2020-06-29T04:32:15.024Z|07396|timeval|WARN|faults: 4 minor, 0 major
2020-06-29T04:32:15.024Z|07397|timeval|WARN|context switches: 14 voluntary, 1 involuntary
2020-06-29T04:32:15.024Z|07398|poll_loop|INFO|Dropped 1 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-29T04:32:15.024Z|07399|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/ovn/ovnsb_db.sock) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T04:32:19.266Z|07400|timeval|WARN|Unreasonably long 4241ms poll interval (4227ms user, 0ms system)
2020-06-29T04:32:19.266Z|07401|timeval|WARN|faults: 126 minor, 0 major
2020-06-29T04:32:19.266Z|07402|timeval|WARN|context switches: 18 voluntary, 1 involuntary
2020-06-29T04:32:19.266Z|07403|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (100% CPU usage)
2020-06-29T04:32:23.501Z|07404|timeval|WARN|Unreasonably long 4236ms poll interval (4219ms user, 0ms system)
2020-06-29T04:32:23.501Z|07405|timeval|WARN|faults: 4 minor, 0 major
2020-06-29T04:32:23.501Z|07406|timeval|WARN|context switches: 16 voluntary, 7 involuntary
2020-06-29T04:32:23.501Z|07407|coverage|INFO|Dropped 13 log messages in last 55 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-29T04:32:23.501Z|07408|coverage|INFO|Skipping details of duplicate event coverage for hash=1a8dc4b6
2020-06-29T04:32:27.754Z|07409|timeval|WARN|Unreasonably long 4252ms poll interval (4238ms user, 0ms system)
2020-06-29T04:32:27.754Z|07410|timeval|WARN|faults: 78 minor, 0 major
2020-06-29T04:32:27.754Z|07411|timeval|WARN|context switches: 10 voluntary, 3 involuntary
2020-06-29T04:32:27.754Z|07412|poll_loop|INFO|Dropped 1 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-29T04:32:27.754Z|07413|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/ovn/ovnsb_db.sock) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T04:32:31.996Z|07414|timeval|WARN|Unreasonably long 4242ms poll interval (4227ms user, 1ms system)
2020-06-29T04:32:31.996Z|07415|timeval|WARN|context switches: 18 voluntary, 1 involuntary
2020-06-29T04:32:31.996Z|07416|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (100% CPU usage)
2020-06-29T04:32:36.253Z|07417|timeval|WARN|Unreasonably long 4258ms poll interval (4243ms user, 0ms system)
2020-06-29T04:32:36.253Z|07418|timeval|WARN|faults: 92 minor, 0 major
2020-06-29T04:32:36.253Z|07419|timeval|WARN|context switches: 12 voluntary, 3 involuntary
2020-06-29T04:32:36.254Z|07420|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (<->/var/run/ovn/ovnnb_db.sock) at lib/stream-fd.c:157 (101% CPU usage)
2020-06-29T04:32:40.547Z|07421|timeval|WARN|Unreasonably long 4294ms poll interval (4279ms user, 0ms system)
2020-06-29T04:32:40.547Z|07422|timeval|WARN|context switches: 18 voluntary, 2 involuntary
2020-06-29T04:32:44.777Z|07423|timeval|WARN|Unreasonably long 4229ms poll interval (4213ms user, 1ms system)
2020-06-29T04:32:44.777Z|07424|timeval|WARN|faults: 62 minor, 0 major
2020-06-29T04:32:44.777Z|07425|timeval|WARN|context switches: 11 voluntary, 3 involuntary
2020-06-29T04:32:44.777Z|07426|poll_loop|INFO|Dropped 1 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-29T04:32:44.777Z|07427|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (100% CPU usage)

4. After the pressure test, use perf strace to track ovn-northd, the output as follows
     0.046 ( 0.004 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
     0.068 ( 0.028 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
     0.101 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
     0.107 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 141) = 141
     0.129 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 108) = 108
     0.134 ( 0.006 ms): poll(ufds: 0x5560db42dbf0, nfds: 4                                    ) = 0 Timeout
     0.141 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
     0.147 ( 0.012 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 140) = 140
     0.165 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 113) = 113
     0.511 ( 0.014 ms): sendto(fd: 14<socket:[552516759]>, buff: 0x55611315ff60, len: 41      ) = 41
     0.528 ( 0.002 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d4e4, size: 1532 ) = -1 EAGAIN Resource temporarily unavailable
     0.533 ( 0.002 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e1fe, size: 2914 ) = -1 EAGAIN Resource temporarily unavailable
  4095.413 ( 0.021 ms): accept(fd: 12<socket:[552516757]>, upeer_sockaddr: 0x7ffef8a0de60, upeer_addrlen: 0x7ffef8a0de5c) = -1 EAGAIN Resource temporarily unavailable
  4095.458 ( 0.003 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
  4095.477 ( 0.025 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
  4095.508 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 70) = 70
  4095.514 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 68) = 68
  4095.519 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
  4095.525 ( 0.005 ms): poll(ufds: 0x5560e3a55ad0, nfds: 4                                    ) = 1
  4095.531 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
  4095.535 ( 0.003 ms): fstat(fd: 14<socket:[552516759]>, statbuf: 0x7ffef8a0da60             ) = 0
  4095.540 ( 0.002 ms): getsockname(fd: 14<socket:[552516759]>, usockaddr: 0x7ffef8a0d970, usockaddr_len: 0x7ffef8a0d8f8) = 0
  4095.545 ( 0.003 ms): getpeername(fd: 14<socket:[552516759]>, usockaddr: 0x7ffef8a0d970, usockaddr_len: 0x7ffef8a0d8f8) = 0
  4095.556 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 150) = 150
  4095.564 ( 0.006 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d4e4, size: 1532 ) = 38
  4095.586 ( 0.012 ms): sendto(fd: 15<socket:[554039164]>, buff: 0x5560efee8e80, len: 41      ) = 41
  4095.601 ( 0.002 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e1fe, size: 2914 ) = -1 EAGAIN Resource temporarily unavailable
  8211.797 ( 0.023 ms): accept(fd: 12<socket:[552516757]>, upeer_sockaddr: 0x7ffef8a0de60, upeer_addrlen: 0x7ffef8a0de5c) = -1 EAGAIN Resource temporarily unavailable
  8211.967 ( 0.005 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
  8211.992 ( 0.025 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
  8212.023 ( 0.004 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
  8212.030 ( 0.005 ms): poll(ufds: 0x5560df0002e0, nfds: 4, timeout_msecs: 883                ) = 1
  8212.037 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
  8212.044 ( 0.003 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d50a, size: 1494 ) = -1 EAGAIN Resource temporarily unavailable
  8212.050 ( 0.006 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e1fe, size: 2914 ) = 38
 12381.888 ( 0.021 ms): accept(fd: 12<socket:[552516757]>, upeer_sockaddr: 0x7ffef8a0de60, upeer_addrlen: 0x7ffef8a0de5c) = -1 EAGAIN Resource temporarily unavailable
 12381.931 ( 0.003 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
 12381.953 ( 0.028 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
 12381.987 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 70) = 70
 12381.993 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
 12381.998 ( 0.004 ms): poll(ufds: 0x5560e3a526d0, nfds: 4                                    ) = 0 Timeout
 12382.004 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
 12382.010 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 140) = 140
 12382.018 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 113) = 113
 12382.386 ( 0.015 ms): sendto(fd: 14<socket:[552516759]>, buff: 0x5560f94f4ef0, len: 41      ) = 41
 12382.406 ( 0.003 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d50a, size: 1494 ) = -1 EAGAIN Resource temporarily unavailable
 12382.414 ( 0.002 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e224, size: 2876 ) = -1 EAGAIN Resource temporarily unavailable

5. After the pressure test, use top to view the process utilization
Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.7 us,  0.7 sy,  0.0 ni, 96.5 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem : 257572.1 total, 184899.2 free,  40058.3 used,  32614.6 buff/cache
MiB Swap:  16384.0 total,  16384.0 free,      0.0 used. 211729.8 avail Mem
PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 75974 openvsw+  10 -10 2107672   2.0g   3116 R  99.3   0.8 202:51.65 ovn-northd
 75949 openvsw+  20   0<PHONE_NUMBER>48   4924 S   0.0   0.1  30:20.48 ovsdb-server
 75965 openvsw+  20   0 2776408   2.5g   4860 S   0.0   1.0   2:11.96 ovsdb-server

I would like to know the possible reasons. Does everyone have any ideas?
Does ovn-northd need parameter tuning？
@putnopvut @numansiddique

@zhanrox2 did you manage to figure this out? we seems to be hitting similar issues too.

I also got the same thing. Mind to share the workaround?


How to pass rustflags / static compile ?

Hello. I would like static compile libc with my program.
How do it?
Cross.toml
[target.arm-unknown-linux-gnueabihf]
image = "cross-armhf:lastest"
volumes = [
    "BUILD_DIR",
    "RUSTFLAGS"
]
rustflags = [
    "-C", "link-arg=-static",
    "-C", "target-feature=+crt-static"
]

Hi hanusek,
as far as I know you have 2 options:
you could just set them with the cargo configuration file described here https://doc.rust-lang.org/cargo/reference/config.html.
and create a config file .cargo/config.toml.
Or If you don't want to use the config, but pass the RUSTFLAGS env variable to the container, you should set them in your shell that you are calling cross from. You can then remove the rustflags entry from Cross.toml.
You have to use the passthrough key in Cross.toml and add RUSTFLAGS there. Note that no values are set in cross but only the variable name that will then be set to the value that are set in your shell (passed through).

Hi hanusek,
as far as I know you have 2 options: you could just set them with the cargo configuration file described here https://doc.rust-lang.org/cargo/reference/config.html. and create a config file .cargo/config.toml.
Or If you don't want to use the config, but pass the RUSTFLAGS env variable to the container, you should set them in your shell that you are calling cross from. You can then remove the rustflags entry from Cross.toml. You have to use the passthrough key in Cross.toml and add RUSTFLAGS there. Note that no values are set in cross but only the variable name that will then be set to the value that are set in your shell (passed through).

Unfortunately, this won't work, since RUSTFLAGS is ignored for cross-compilation. We've documented how to configure cargo when cross-compiling in the wiki.


An option to add unique text sounds for characters

What feature do you want to get added on the base engine?
What if the Dialogue Portrait Editor let you give each character a unique "text typing" sound instead of always using the default?
Here's a quick mockup I made in GIMP of what it would look like:

maybe an issue that looks like #332


Feature request: CollaborationPlugin

Thanks for making Lexical available to Svelte!
I can understand if this will take a lower priority, but I'd love to see support for the <CollaborationPlugin>. Yjs makes realtime collaboration and offline storage very easy, and Lexical has support for Yjs out of the box:
https://lexical.dev/docs/collaboration/react
If it's relatively straight forward, I could give it a try myself. If so, I would just need some pointers to get started. I'm familiar with Svelte, but not yet with Lexical.

Hi @joakim ,
I have been focusing on the core editor features and have been avoiding the work of collaboration. We have made significant progress but I still have work to do with a number of plugins.
I would really appreciate it if you could assist with implementing the CollaborationPlugin. I believe, the first step would be to have a working version of collaboration in lexical itself. I can't find it in the playground.
If you are able to contribute I would suggest starting with building a collaborative editor in React Lexical. The link you shared above is a good starting point.

Nice, thank you! Collaboration is a big part of my project, so I really appreciate this. I'll definitely move over to svelte-lexical, as soon as I have the time :)


Training stops after 1 epoch for RCNN

Hi all,
I could complete the training for RPN successfully. However, when I train the RCNN network using the model created in the previous step, the training stops after 1 epoch. Could you please help me figure out as to what could be the issue?
Thanks in advance.
-Nikhitha

no disk space left?


Input ops fed networks operate considerably slower than direct feed ones

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
http://stackoverflow.com/questions/39794149/when-introducing-the-optimizer-variables-under-variable-scope-get-recreated-twic
Environment info
Tried both OS X and Linux (Ubuntu 16)
On both utilizing CPU only.
If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)
Any simple operation (i.e. calculating logits with a 3 layer deep network with simple regression in each layer) in which the data was fed with either a parse_example, parse_single_example, building a CustomRunner that feeds a RandomShuffleQueue or utilizing QueueRunner.  In all of those the operation and time to complete 1 epoch took considerably more than if I were to save the data in a bumpy array and feed it during a call to sess.run() with the feed_dict.
What other attempted solutions have you tried?
Tried to score the web for solutions and ran into a couple blog posts describing the problem but can't seem to find solutions.
Logs or other output that would be helpful
Tried this on both TF 0.10 and the latest RC for 0.11.

@ebrevdo might have some thoughts on this.

I looked at some timelines in https://github.com/tensorflow/tensorflow/issues/4740 and here.
On official MNIST examples QueueDequeueMany is the bottleneck, whether it's reading using parse_single_example or using slice_input_producer, with or without the GPU, taking 10x longer than entire duration of step when using feed_dict with same data size

examples/how_tos/reading_data/fully_connected_reader.py
examples/how_tos/reading_data/fully_connected_preloaded.py
examples/how_tos/reading_data/fully_connected_preloaded_var.py

(see #4740 for an attached timeline)

QueueDequeueMany takes a long time when it's waiting for enough stuff to get put on the queue.  It's not computation cycles that are slowing it down.
You need multiple reader threads all feeding the queue in order to unblock it.  If you're using tf.train.batch or its friends, in TensorBoard you'll see that the queue is never full (and it should be).  Increase the number of threads you use; this is a simple argument in tf.train.batch and friends.

Ah, good point, increasing capacity, number of threads, and adding time.sleep in main thread to preload the examples makes preloaded_reader example as fast as feed_dict one.
https://github.com/yaroslavvb/stuff/commit/226099cd883df5aa5b08483581d66960ada90e22
Note that simply increasing capacity/number of threads didn't help. I suspect Python just ran the main queue until it ran out of examples, and went back to slow behavior with blocking (Python3 specific thread behavior maybe?).
Also, seems that there's no way to increase number of threads in slice_input_producer

@ebrevdo in my case, when producing timelines for with and without input ops, vs, feed_dict, it seems that the majority of the time is spent on a matmul operation in my hidden_layer_1_1 which is the shared scope of my test.  (I obviously share my weights and biases in the hidden layers between my training data and my test data).
That operation, when used with input ops variables, takes 743ms!!!!!!

It appears that the MatMul operation takes a really long time since my test data size is longer than the training data batch size. (Training data is 100 example at each run while test is 3800 examples)
The question is why is sharing variables causing this size of a test data run to take so long as compared to utilizing place holders and not using variables scope sharing.
Also important to note that overall with the smaller test batch size and variable sharing I'm still seeing about 100ms per run compared to 75ms per run with placeholders and a longer test size batch (3800 examples).
But that's not as bad as my main problem which is 1000ms vs about 100ms

The situation is even worse when utilizing parse_single_example vs a QueueRunner with data preloaded and fed that way into the QueueRunner.
With MatMul still taking a long time (for the reason stated in the above comment) and now in addition to that QueueDequeueMany taking a good 360ms. And that's with shuffle_batch, 4 threads, batch size of 100, capacity of 1000 and min dequeue of 200.

Just to add some info, my features tensor is about 32,000 features.  So the shape of the test set is (3800,32000) and the training data is usually of shape (100,32000).
Like i said, doing the above utilizing feed_dict takes about 80ms per sess.run() call and utilizing a QueueRunner with directly fed data items from numpy array (only for the training data, the test data is a const with tf.convert_to_tensor()) takes about 1000ms total with that matmul taking most of the time.  And finally utilizing a two tfrecord files, one for the training data and one for the test data, and feeding with parse_single_example for the training data and parse_example for the test data, QueueDequeueMany (<- of the test data, so the one using parse_example) and Matmul are the two highest time consumers bringing the sess.run() up to around 1800ms.

It sounds a bit suspicious that matmul itself would take different time depending on feed_dict vs Queue. Are you getting MatMul timing from the Timeline? Are you sure that matmul sizes are the same? Unlike with QueueDequeue many, IO time should not be included in the timeline timing, so if size is the same, the timing should also be the same
Here's how you can generate text file with timing information. In that file you can see duration of matmul as well as total output size for sanity checking
    node_stats {
      node_name: "hidden1/MatMul"
      all_start_micros:<PHONE_NUMBER>315055
      op_start_rel_micros: 1
      op_end_rel_micros: 314
      all_end_rel_micros: 318
      memory {
        allocator_name: "cpu"
        total_bytes: 51200
      }

If the this timing information shows same total_bytes but 10x difference in timing, the way to get this triaged would be to provide a minimal self-contained reproducible test case

@yaroslavvb i'm certain about the size being the same.  But the difference between the two runs, except for being feed_dict vs Queue is that the matmul operation that takes long utilizing Queue is happening on the shared variable op node, an op that didn't exist (since it wasn't needed) on the feed_dict run.
I got the time from timeline, yes.  And i assume the time is what's called wall time.
Finally, I'll go ahead and run it again, and save the raw RunMetadata this time.  I'll follow up with another comment shortly with the results of that.

@yaroslavvb so doing that revealed something I didn't realize before and that now i'm not sure how to proceed with.
First things first, there's no problem with Matmul.  It produces the same amount of execution time for the same amount of bytes. I recorded only the first example from the two below, which in the feed_dict run didn't include the test data run.  I found that out by looking at the dim size and the output bytes as you suggested.
Basically I do something like the following:
sess.run([optimizer, cost, summeryMerge])
And later on I do something like accuracy.eval()
In the Queue based run, the value of accuracy is calculated at every run, even without me calling accuracy.eval() - I'm assuming because of the shared variables being updated since the graph has them built in.  That in turn is why my runs are considerably slower overall compared to feed_dict.  The operation takes the same amount of time, but is happening at every batch now instead of once per epoch.  I'm assuming that's also why the Queue that holds the test data gets depleted fast and takes a long while to continue producing sample data, causing QueueDequeueMany to appear slow.
My question then is how does one go about using Queues and feeding in test data but have it computed only once per epoch instead of at every batch run.  From reading Sharing Variables, I thought thats the way to go with training/test data.  So I have a structure similar to the following code:
def neural_network_model(x):
    with tf.variable_scope("hidden_layer_1"):
        relu1 = hl_relu(x, [max_words_len, n_nodes_hl1], [n_nodes_hl1]) #hl_relu utilizes get_variable for weights and biases

And later on i'd do something like this:
def train_neural_network(x_batch, y_batch, test_x, test_y):
    with tf.device("/cpu:0"):
        with tf.variable_scope("dnn") as scope:
            logits = neural_network_model(x_batch)
            scope.reuse_variables()
            test_logits = neural_network_model(test_x)

I'd use logits to compute the cost with softmax_cross_entropy_with_logits, as follows:
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits, y_batch), name="cost")

and test_logits to compute the accuracy in a three step operation looking something like this:
prediction = tf.nn.softmax(test_logits)
correct = tf.equal(tf.argmax(prediction, 1), tf.argmax(test_y, 1), name="correct")
accuracy = tf.reduce_mean(tf.cast(correct, tf.float32), name="accuracy")

What am I doing wrong?

Maybe i should just simplify my question.  How do I train and test in the same process, using shared variables with 2 tfrecord files, one designated as train data and one designated as test data.
As mentioned briefly in on the last line of https://www.tensorflow.org/versions/r0.11/how_tos/reading_data/index.html#multiple-input-pipelines

I'm confused why matmul timing would change unless the shapes changed, or
perhaps if it for some reason started running on a CPU device where before
it was running on a GPU device.  Please provide the runmetadata.
On Wed, Oct 5, 2016 at 8:11 PM, Daniel Schonfeld<EMAIL_ADDRESS>wrote:

Just to add some info, my features tensor is about 32,000 features. So the
shape of the test set is (3800,32000) and the training data is usually of
shape (100,32000).
Like i said, doing the above utilizing feed_dict takes about 80ms per
sess.run() call and utilizing a QueueRunner with directly fed data items
from numpy array (only for the training data, the test data is a const with
tf.convert_to_tensor()) takes about 1000ms total with that matmul taking
most of the time. And finally utilizing a two tfrecord files, one for the
training data and one for the test data, and feeding with
parse_single_example for the training data and parse_example for the test
data, QueueDequeueMany (<- of the test data, so the one using
parse_example) and Matmul are the two highest time consumers bringing the
sess.run() up to around 1800ms.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/4732#issuecomment-251856881,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABtim3mQAeLWGCZCOdnkF1K4ylL5ey5sks5qxGbNgaJpZM4KMz--
.

@ebrevdo the shape changed... it was my mistake.  My timing comparison was incorrect.  When compared with the same shape the timing was identical.
So now I'm just left wondering, what I asked above.  How can I do both train and eval in the same process using input ops?

It's atypical to do this.  Folks usually build two graphs: one for
training, and a separate one for eval.  (you can use a common function to
build the "common" subgraph).  You then run the train and eval graphs
either in separate threads or in different processes.  At the very least,
you use different sessions for training and eval.
On Thu, Oct 6, 2016 at 9:08 AM, Daniel Schonfeld<EMAIL_ADDRESS>wrote:

@ebrevdo https://github.com/ebrevdo the shape changed... it was my
mistake. My timing comparison was incorrect. When compared with the same
shape the timing was identical.
So now I'm just left wondering, what I asked above. How can I do both
train and eval in the same process using input ops?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/4732#issuecomment-252010220,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABtim2jldBDZ9pE6HgkqTgkQ96TVIcwCks5qxR0bgaJpZM4KMz--
.

@ebrevdo I see... so i should adopt more of like the cifar-10 example model where one process trains and one monitors for checkpoint files and loads and tries the eval stuff against?

Exactly. Then you keep the best checkpoint.
On Oct 6, 2016 9:25 AM, "Daniel Schonfeld"<EMAIL_ADDRESS>wrote:

@ebrevdo https://github.com/ebrevdo I see... so i should adopt more of
like the cifar-10 example model where one process trains and one monitors
for checkpoint files and loads and tries the eval stuff against?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/4732#issuecomment-252014914,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABtimw_dRn0cy_ytHS9OTOP9p_IeJVNRks5qxSD4gaJpZM4KMz--
.

now if only i can figure out why my raw usage of tensorflow only achieves 60% accuracy compared to using the SciKit module of TF DNNClassifier which archives like 99% pretty fast.
Perhaps the usage of embedded lookup? We can probably close this issue for now, unless someone else wants to keep it alive?

Going through the issues mentioning r0.11.
Looks like the initial issue is resolved here. Closing.


Useful examples

Need some illuminating examples. perhaps

static assets built at compile time.
examples on top of raxx

An in memory static file server.
If memory too large create multiple releases and have a hashing algorithm to nodes.
Similar functionality to udon

Hello cluster,

multicast for node discovery
suspend startup until minimum  number of nodes connected.
show different content based on either ip address, or release arguments or environment.
Show links to same page at other ips, system should ping release to ensure release is also a greeting cluster.
Use server sent events to update the list of nodes that are online.

Found in Tokumei


No possible way to set temperatur in home assitant

Hey,
First of all thank you a lot for the work you are putting into this project!
I'm able to get a lot of information and configure some parameters for my Marvel R, but I don't have any possible way to set the temperature for A/C only turn it on and off.

Hello, same issue with domoticz
"Error sending switch command, check device/hardware (idx=95) !"
Logs :
2024-02-25 11:41:18.593 Error: MQTT Disovery: climate device unhandled mode_state_template (***_vehicle_climate/MG Marvel R (Climatisation))

Same here for my MG4.

Please try 0.6.0-rc2

@nanomad Shouldn't this be:
        if self.__remote_ac_temp is not None:
            self.set_ac_temperature(self.__remote_ac_temp)

I actually manage to solve it before I updated to 0.6.0-rc2 by adding the following to the state attribute under developer tools -> state->find the climate entiti (climate.lsjeXXXXXXXXXXX_vehicle_climate):
temperature: 25
that made I possible to change what temp the is set when set the HVAC mode to auto

i tried deleting the state attribute updating the gateway and it now configured again. so looks like I solved it

I also tried the prerelease version and it's working as expected now. Thank you!


Linter: Next Error jumps to seemingly random error

"next" should be relative to the cursor, but I think it's jumping based on some internal ordering or internal caching.

It should be super-easy to keep track of last error once we land https://github.com/atom-community/linter/pull/827

It should be super-easy to look at where the cursor is and find the next error. Keeping track of it after you land is definitely the wrong approach.
I literally want to jump to the first lint error in my file, so I jump to the top of the file, then trigger Linter: Next Error. As it currently is, I go to some random lint error in the middle of the file.

Right, our current sort function could be the problem. We could sort using the ranges.

https://github.com/atom-community/linter/blob/master/lib/commands.coffee#L71-L75
It isn't even looking at where the cursor is. It assumes the first time you call it you want the first error, and every time after that you want the next index no matter what has changed.

+1

This will be fixed in https://github.com/atom-community/linter/issues/865

Actually this is fixed in #851

Agree that it fixes it, but #865 fixes it throughout the package.
If we do that, we won't have to sort anything in commands.


Timeout attribute is ignored

While executing this:
\DB::connection('mongodb')->collection('stats')->where('updated_at','>=', $from)->where('updated_at','<=',$to)->timeout(-1)->count()
[ $from & $to are DateTime objects ]
I still get MongoCursorTimeoutException after 30 seconds, hence mongo is ignoring the timeout setting on the query builder.

Hi,
Any ETA on this bug?

Hi,
unfortunately this bug is still open. Any updates?


Update PagerDuty scheduler to respect weekends

Description
As discussed in the engineering sync on 10/14, we'd like to shift the pagerduty schedule such that the primary oncall is primary through the entire weekend (instead of the current schedule, where the secondary is paged first from 9AM-6PM EST)
Proposed solution
Change the PagerDuty rotations so that one person is primary Friday evening -> Monday morning, and one person is the secondary during the same period.
Additional context
https://docs.google.com/document/d/1iBRku5Jp2rQ9IC8SkDAMwf_MWspv8-ArxI-1JgnaSJc/edit#

Closing ticket due to the current schedule on call schedule.


bugs - root username / cannot reinstall

Debian for Windows at the first run after installation asks for username. It allows enter root username, then after entering password errors will appear. Next run console appears just fine, simple commands like ps or w works, but even apt-get update do not work and shows errors. After Debian uninstall, Windows restart and install Debian again I tried another username, and this appeared:

I suspect that uninstall is not full and some settings/files have not been deleted. So, what should I do, to make a clean install?

Try to uninstall with wslconfig /u debian.


setState locks seem to sometimes fail to be released when an error is thrown

I frequently end up with stack traces like this one:
20:03:10.872 - Plugin failed to run: ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.Properties:73: attempt to get length of field 'instances' (a nil value)
20:03:10.902 - ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.ComponentData:11: setState cannot be used in the render method.
20:03:10.902 - render must be a pure function that only depends on props and state.
20:03:10.903 -
20:03:10.903 - Check the definition of render in the component "ComponentData".
20:03:10.903 - Stack Begin
20:03:10.903 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Roact.Component', Line 225 - method setState
20:03:10.904 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.ComponentData', Line 11 - local func
20:03:10.904 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Signal', Line 36 - method fire
20:03:10.904 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.ComponentManager', Line 165 - method doUpdate
20:03:10.905 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.ComponentManager', Line 173
20:03:10.905 - Stack End
20:03:11.186 - ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.ComponentData:11: setState cannot be used in the render method.
20:03:11.187 - render must be a pure function that only depends on props and state.
20:03:11.187 -
20:03:11.187 - Check the definition of render in the component "ComponentData".
20:03:11.188 - Stack Begin
20:03:11.188 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Roact.Component', Line 225 - method setState
20:03:11.188 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.ComponentData', Line 11 - local func
20:03:11.188 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Signal', Line 36 - method fire
20:03:11.189 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.ComponentManager', Line 165 - method doUpdate
20:03:11.189 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.ComponentManager', Line 173
20:03:11.189 - Stack End
20:03:11.353 - ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.ComponentData:11: setState cannot be used in the render method.
20:03:11.353 - render must be a pure function that only depends on props and state.
20:03:11.354 -
20:03:11.354 - Check the definition of render in the component "ComponentData".
20:03:11.354 - Stack Begin
20:03:11.354 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Roact.Component', Line 225 - method setState
20:03:11.355 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Components.ComponentData', Line 11 - local func
20:03:11.355 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.Signal', Line 36 - method fire
20:03:11.355 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.ComponentManager', Line 165 - method doUpdate
20:03:11.356 - Script 'ReplicatedStorage.ComponentVisualizerPlugin.Plugin.ComponentManager', Line 173
20:03:11.356 - Stack End

As far as I can tell, every single one of these errors that setState() cannot be used in the render method are red herrings caused by the initial true error at the top. The stack trace there is within the render() method.
What I believe is happening is that Roact lifecycle code sets the lock, then calls my render method (without using pcall), which fails, and then the lock is never released because that code never runs. Then, when code later down the line calls setState(), it fails because the lock is still held.
I'm pretty sure this is actually what's happening because this is a functional component and I can see that the reconciler does this while mounting functional components:
		local vdom = element.component(element.props)

This is spot-on; there's an open issue in #84.
I was afraid of the potential overhead of using xpcall for every piece of user code; do you think it would be small enough to use everywhere?

I wrote a quick benchmark to see what the relative cost is:
local function f(x)
    return 4 + x
end

local start = tick()
for i = 1, 10000 do
    local r = f(i)
end
print(tick() - start)

local start = tick()
for i = 1, 10000 do
    local r = xpcall(f, i)
end
print(tick() - start)

-- results:
-- 0.00059890747070313
-- 0.057153701782227

That's a lot worse than I was hoping for... I'm not sure what the best path forward is here. Going from the approaches suggested in the other issue, I think I prefer approaches where we somehow reset the locks after recovering from an error, but I don't know how that'd work.
Feel free to close this one as a duplicate of the other one.

New reconciler can help with this a bit, since the pcall can be on the task level at least.
Another issue is that the tree can be an arbitrarily screwed up state unless we're really careful to make the reconciler fairly atomic between user call invocations.
This is something that React doesn't really have to contend with since try ... catch in JS is pretty much free.


[EngSys] Update central Service Bus version

Summary
The focus of these changes is to update the central version for the Azure.Messaging.ServiceBus package to use the latest release.

Failure is an unrelated flaky test exceeding the timeout.  Merging.

/check-enforcer override


Redirect to previous page if asked for password

Currently if you get prompted for a password, you get redirected to the homepage instead of the page you were currently on.

I started looking at this. Log into lbry.tv, it pulls and populates your sync settings. But when you go to push sync settings, you don't have your wallet password, it asks for that, then redirects you through the first run flow choosing tags and channels to follow.


Create table_pagination_h2o3.py

Based off of the table_pagination_pandas.py example but in theory this should work for any h2o-3 DataFrame with very minimal changes to the code. All changes end-users may want to make are marked with a TODO.
I am not sure if other files need to be updated when you make a new example.

Merging this so that I could merge the server-side table PR. We can deal with minor improvements later as needed.


Add Unpublish tab to the edition editor

This adds unpublish (fomerly know as 'withdraw') functionality to Publisher.  This functionality is copied from Panopticon, where it is soon to be deleted.
It's 'old world' publishing platform, in that it expects to be managing routes and
Rummager content itself, rather than letting the publishing-api manage them.
When we move the mainstream formats to a new Schema (soon!) we will need to revisit
this to stop it from speaking to the router-api and rummager directly.
Trello: https://trello.com/c/xM4HRRYt/574-preparatory-work-moving-withdrawing-functionality-into-publisher-5

Paired with @whoojemaflip

What happens when the user tries to unpublish an already unpublished artefact? Is there a warning message of some sort (sorry Integration is busy at the moment, can't test it myself yet). Should we add an integration test for that?

@davidbasalla Re: unpublishing an already unpublished artefact, we do cover that in lines 38-43 of unpublish_test.rb.  Do you think it would be clearer to split it out into another test?

@emmabeynon I guess I was more thinking about what happens when unpublishing fails? Looks like a flash message gets set -> https://github.com/alphagov/publisher/pull/528/files#diff-3d0014943f4437aab728020d24b993ebR195 We could add an integration test for that?

@davidbasalla Now added a test in for redirecting to an external link.  This involved a change to setting the flash alert in the editions controller, which should also address your point about what happens if unpublishing fails.


03/28/2018 06:58 : Create Pull Request for branch master -> prTesting1_2018032806

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestCon1.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Non-existing input file doesn't result in error

It would be nice if ts2ocaml warned or errored if given a non-existing file as input.
Currently ts2ocaml jsoo --create-minimal-stdlib --preset=minimal --safe-arity=full --simplify=all not-existing-file.d.ts returns 0, generates the minimal stdlib, but does nothing else to warn the user.
Making sure the input files have correct extension .d.ts may also be nice.

Not reproducing in my environment:
Error: ENOENT: no such file or directory, open 'not-existing-file.d.ts'
    at Object.openSync (fs.js:498:3)
    at Object.readFileSync (fs.js:394:35)
    at /usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:381811
    at /usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:33961
    at xn.next (/usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:31572)
    at xn.System.Collections.IEnumerator.MoveNext (/usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:31140)
    at /usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:33955
    at xn.next (/usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:31572)
    at xn.System.Collections.IEnumerator.MoveNext (/usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:31140)
    at en (/usr/local/lib/node_modules/@ocsigen/ts2ocaml/dist/ts2ocaml.js:2:27744) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: 'not-existing-file.d.ts'
}

It should fail on fs.readFile().
I will add another layer of existence & file extension check regardless.

Maybe it depends on the version of Node. Can you let me know which version of Node you are using?

I think I figured out why: TypeScript compiler API doesn't throw an exception if the file actually doesn't exist. I need to add a manual check here.


Directly link Sentinel license

Closes #290
This PR:

Links Sentinel license directly to https://sentinels.copernicus.eu/documents/247904/690755/Sentinel_Data_Legal_Notice for S1-GRD, S2-L1C /L2A, S3-OLCI/SLSTR and S5P

You still missed setting the correct Type for the License link object. I pushed a fix for this. See if you understand what I did here (the statement basically checks if LicenseUrlType is present in the yaml file and if so it sets the specified LicenseUrlTpye. If it's not set then we set the default "text/html") and then we can merge this.


daikin brp069a62 no details

have installed a daikin brp069a62 on to a daikin altherma monblock system with a room thermostat
unit comes on line and is seen by home assistant
but only the Hot water cylinder is seen and all the sensors are off line
know issue ?
is it a firmware update required on the daikin brp069a62 if so can not see any firmware on the daikin website
thankyou

I had the same problem as you for months, but this week I got a firmware update to the gateway and it seems to be solved.
Please install the firmware update proposed by the “ONECTA - Daikin” App. The new version is 436CC180000.

Is there a way to get the firmware manually?
On Mon, 29 Apr 2024, 09:00 lufetico, @.***> wrote:

I had the same problem as you for months, but this week I got a firmware
update to the gateway and it seems to be solved.
Please install the firmware update proposed by the “ONECTA - Daikin” App.
The new version is 436CC180000.
—
Reply to this email directly, view it on GitHub
https://github.com/tadasdanielius/daikin_altherma/issues/92#issuecomment-2082100607,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BE74EDKZ42WHV73PAJTWEFLY7X4YDAVCNFSM6AAAAABGQNSA3SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBSGEYDANRQG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

It is probably possible, but I don't know how to do it. Maybe in a Daikin developer forum they can help you.

I spoke too soon. Today the integration started to fail again. :(

Interesting my daikin updated to day the controller
Not yet tried the lan gateway. This was via the WiFi adapter
On Mon, 29 Apr 2024, 20:33 lufetico, @.***> wrote:

I spoke too soon. Today the integration started to fail again. :(
—
Reply to this email directly, view it on GitHub
https://github.com/tadasdanielius/daikin_altherma/issues/92#issuecomment-2083511277,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BE74EDPDI2TDCDHMRVMAADDY72N7BAVCNFSM6AAAAABGQNSA3SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBTGUYTCMRXG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

Contracted Daikin . They advised have another attempt at connecting the adaptor. This time it did 2 firmware updates , and after another power cycle . I now have connections into home assistant
Daikin adapter on version  436CC180000.
Apparently firmware is not available to download and install manually

See above


Sparse model crashing at certain input

Hey Gerogy,
Awesome and clean project! I am using rainymotion in my MSc thesis about weather nowcasting for intraday electricity trading. It works pretty neat for benchmarking optical flow methods.
My issue is that the Sparse model crashes on certain input.
Minimum example code:
import glob

import numpy
from PIL import Image
from rainymotion.models import Sparse

imageFolderPath = f'/PATH/TO/failing_data'
imagePath = sorted(glob.glob(imageFolderPath + '/*.png'))

im_array = numpy.array([numpy.array(Image.open(img).convert('L'), 'f') for img in imagePath])
model = Sparse()

model.input_data = im_array
nowcasts = model.run()

failing_data.zip
My investigations went until _sparse_linear in models.py,
new_corners, st, err = cv2.calcOpticalFlowPyrLK(prevImg=data_instance[i - 1],
                                                nextImg=data_instance[i],
                                                prevPts=old_corners,
                                                nextPts=None,
                                                **of_params['lk_pars'])

This method call returned all None-s and causing crashing 1 line after.
All output:
/home/user/anaconda3/envs/rainymotion/lib/python3.7/site-packages/wradlib/io/xarray.py:951: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  class XRadVol(collections.MutableMapping):
Traceback (most recent call last):
  File "/home/user/Workspace/Dexter/task-pod-ml/nowcasting/src/optical_flow_sparse_error_minimal.py", line 15, in <module>
    nowcasts = model.run()
  File "/home/user/anaconda3/envs/rainymotion/lib/python3.7/site-packages/rainymotion-0.1-py3.7.egg/rainymotion/models.py", line 256, in run
  File "/home/user/anaconda3/envs/rainymotion/lib/python3.7/site-packages/rainymotion-0.1-py3.7.egg/rainymotion/models.py", line 60, in _sparse_linear
AttributeError: 'NoneType' object has no attribute 'ravel'

I have similar inputs like this, but they do not crash all the time just with certain sets. All other models work fine with the same data.

Hi Elekes,
sorry for the late response -- there are Easter holidays in Germany from Friday until today.
Thanks for reporting, I will try to check your issue tomorrow when coming back to my work :-/
Best,
Georgy

Hi Elekes,
The problem you described relates to data preprocessing routine. As 'rainymotion' was developed primarily as a tool for rainfall nowcasting, it has two built-in methods for data conversion from rainfall depth or reflectivity to 'uint8' type (0-255 integers) to be compatible with computer vision algorithms provided by opencv and scikit-image libraries -- namely, .scaler and .inverse_scaler as methods in Sparse() model class.
So, it is important to explicitly assign these methods to the new scalers if you are dealing with different data from RY product of DWD.
As a solution, I propose:
import glob

import numpy
from PIL import Image
from rainymotion.models import Sparse

# I have modified data reading process a bit
#imageFolderPath = f'/PATH/TO/failing_data'
#imagePath = sorted(glob.glob(imageFolderPath + '/*.png'))
imagePath = [f'failing_data/{i}.png' for i in range(24)]

im_array = numpy.array([numpy.array(Image.open(img).convert('L'), 'f') for img in imagePath])

# Put new scalers here
# So we just need to convert 'float32' to 'uint8' for **scaler** and fo nothing for **inverse_scaler**
def PNG_Scaler(X):
    return X.astype("uint8"), None, None

def PNG_inv_Scaler(X, c1, c2):
    return X.astype("uint8")

# create model instance and assign our data and functions
model = Sparse()
model.input_data = im_array
model.scaler = PNG_Scaler
model.inverse_scaler = PNG_inv_Scaler

# now it works
nowcasts = model.run()

By the way, I do not recommend you to use the Sparse group of models. For better performance try to use the Dense group.
Thank you for reporting the problem -- in the next version of rainymotion I will try to make a module for automatic inference of input data type to avoid most of I/O problems users have at the moment.

Thanks a lot, it does help with the issue, however still not for everything.
I have similar satellite imagery, same sized images, 11 different spectral bands. The size and data type (numpy.float32) is the same every case.
Traceback (most recent call last):
  File "/home/user/.pycharm_helpers/pydev/pydevd.py", line 1741, in <module>
    main()
  File "/home/user/.pycharm_helpers/pydev/pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/user/.pycharm_helpers/pydev/pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/user/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/user/tmp/pycharm_project_666/nowcasting/src/optical_flow.py", line 126, in <module>
    nowcasts = model.run()
  File "/home/user/anaconda3/envs/rainymotion/lib/python3.7/site-packages/rainymotion-0.1-py3.7.egg/rainymotion/models.py", line 396, in run
  File "/home/user/anaconda3/envs/rainymotion/lib/python3.7/site-packages/rainymotion-0.1-py3.7.egg/rainymotion/models.py", line 133, in _sparse_sd
cv2.error: OpenCV(3.4.4) ../modules/video/src/lkpyramid.cpp:1231: error: (-215:Assertion failed) (npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0 in function 'calc'

And an other, by returning again None in models.py:36.
old_corners = cv2.goodFeaturesToTrack(data_instance[0], mask = None, **of_params['st_pars'])

I found the reason - like which spectral band causing the error - but I don't see a difference from the data storing standpoint. If I find the solution I report it here, or if I can't this week and you have capacity I'll make a more clear description again.

At the moment there is no direct way to use rainymotion for images with more than one channel. We will fix it in the next version.

Converting beforehand the files from RGB to 1 channel grayscale did the trick!
Thanks


rough draft

I've only ever poked at golang, so there might be some very crude mistakes in here.  That said, I'm happy to fix them.  : )

services/kubeapi.go:84: syntax error: missing { after if clause
services/kubeapi.go:85: syntax error: unexpected =, expecting comma or }
FATA[0025] exit status 2
make: *** [ci] Error 1```

In hindsight, I should have checked out the pull request first.


Wildfly 2.0.1

What does this PR do?:
Upgrade WildFly stack

[X] Contributing guide
Have you read the devfile registry contributing guide and followed its instructions?
[X] Test automation
Does this repository's tests pass with your changes?
[ ] Documentation
Does any documentation need to be updated with your changes?
[X] Check Tools Provider
Have you tested the changes with existing tools, i.e. Odo, Che, Console? (See devfile registry contributing guide on how to test changes)

How to test changes / Special notes to the reviewer:

I don't understand the odo failure as i couldn't reproduce locally

/retest


[microNPU] Upgrade to 21.11 version of Arm(R) Ethos(TM)-U55 NPU driver

This PR upgrades the Arm(R) Ethos(TM)-U55 NPU driver to version 21.11
@manupa-arm @ekalda @areusch @Mousius

Prioritizing the bug fix : https://github.com/apache/tvm/pull/9706 over this one. Hope you dont mind :).
Also, we need to mention this in  https://github.com/apache/tvm/issues/9762.

Thanks! @grant-arm @ekalda


Return value from mraa_i2c_read_byte_data not sane.

Does it make sense to return a negative result from a function that is typed as unsigned?  What if I read a register that legitimately returns 0b11111111?  Like say one of the data registers from the MCP23017's on a Numato I/O extender shield?
/**

Read a single byte from i2c context, from designated register

@param dev The i2c context
@param command The register
@return The result of the read or -1 if failed

*/
uint8_t mraa_i2c_read_byte_data(mraa_i2c_context dev, const uint8_t command);
And looking at the implementation, it doesn't look like it matches the API doc either.  It appears the returned value will be 0, not -1, if the call to mraa_i2c_smbus_access fails. Also if I read 0b11111111 (0xFF) from the device (a perfectly legitimate value), then the function returns that value which is -1.  If I then test that value, it would (erroneously) indicate failure right?
Or have I just been awake way too long?
uint8_t
mraa_i2c_read_byte_data(mraa_i2c_context dev, uint8_t command)
{
i2c_smbus_data_t d;
if (mraa_i2c_smbus_access(dev->fh, I2C_SMBUS_READ, command, I2C_SMBUS_BYTE_DATA, &d) < 0) {
    syslog(LOG_ERR, "i2c: Failed to write");
    return 0;
}
return 0x0FF & d.byte;

}
Also applies to mraa_i2c_read_byte and mraa_i2c_read_word_data I would think.

Thanks for the catch! You're entirely right, the doc is outdated, we used to have -1. However I don't understand what you say here:

Also if I read 0b11111111 (0xFF) from the device (a perfectly legitimate value), then the function returns that value which is -1. If I then test that value, it would (erroneously) indicate failure right?

If you read 0xFF, then it'll work just fine. Tbh we don't really need the '0x0FF &' anymore either... That's just 'heritage' from our past use of smbus.c from the roadnarrows i2c lib...
-> 0x0FF & 0xFF
 = 255

Ok in light of the document change how am I to interpret a result of "0"?  An error?  But it isn't an error.  '0' is a legitimate thing to read from an i2c device isn't it?  For instance we use the Numato GPIO extender shield which has 2 MPC20317's (U2 and U4) providing 28 additional bits of GPIO.  They are arranged in 4 separate read data registers of 8 bits each (two, a&b, on each chip, though bits 0..3 of U4 GPIOA are not used).  Now if I connect dry contact sensors to ground on all 8 bit of one of these registers and they all happen to close and I read the register I will get a '0' (0b00000000).  That is not an error!  That again is a legitimate value to read from the device, just as '-1' (0b11111111) is.
The point is that you cannot return an error code in the same 8 bit value you return 8 bits of data. That is one bit of information too much!  It really doesn't matter how you define the failure result. Both 0x00 and 0xFF (i.e. -1) are legal values for an 8 bit register.
Right?

@xthunderheartx agreed, it's not ideal, however I don't want to change types without a bit more thought. Can you detail how you would prefer this to be fixed in #229?

Sure.  I'll code it up for ya and you can look at the delta's and pull if you want.  The ideal solution is to change the functions to work like their reciprocal 'write' operations, i.e. pass the result in the return and the data in an uint8/uint16 pointed to in a formal parameter of the function.  Prototypes would look something like:
mraa_result_t mraa_i2c_read_byte_data(mraa_i2c_context dev, uint8_t data, uint8_t command)
This would break users of the current library but maybe appropriate preprocessor measures could be taken to prevent that. I personally hate a bunch of conditional compilation in my code but sometimes it's needed.  Of course the mraa C++ classes that use these function would need to be fixed as well, but I looked at them and seems straight forward enough.  Like I said I'll code it up, then you can smoke it over and decide.
Thanks Brendan!
P.S. I also like the idea of normalizing the return codes around mraa_result_t, which the reciprocal functions use.

That sounds like a great plan, in C++ we can just throw exceptions. I've got a few API breaks planned for 1.0 so that would fit right in. I guess your protorype would be more this though right?
mraa_result_t mraa_i2c_read_byte_data(mraa_i2c_context dev, uint8_t* data, uint8_t command)

Whilst we're breaking the API i'd rather switch the order of command and data, it feels like command should be first. Look forward to your PR I'll be more than happy to review. Cheers!

Right-o brother.  Forgot the asterisk and I agree about the order of parameters.  I thought the same thing, and if you're gonna break it, break it the right way ;)  I'll get the PR to you as soon as I have a few minutes.  Thanx!


Changing version dependency on jaxlib

This is a trivial change to Line 68 of docker/Dockerfile that has the requirement for jaxlib==1.6.9.
A detailed description is provided in the associated issue.
Closes #298

This needs quotes around the argument, otherwise the > will redirect to a file.
RUN pip3 install --upgrade pip \
    && pip3 install -r /app/alphafold/requirements.txt \
    && pip3 install --upgrade jax "jaxlib>=0.1.69+cuda${CUDA/./}" -f \
      https://storage.googleapis.com/jax-releases/jax_releases.html

Thanks, this was fixed in https://github.com/deepmind/alphafold/commit/57a2455e5fab752d8df430ae1d0334e1a21b548c


Improve the Representation Format for Rules

Currently we use the same rule format for representing rules while parsing and also while planning their execution. This has two problems:

The data structure contains too much detail that is necessary for parsing but superfluous later on
The allowed syntax allows for more flexibility then is required. This results in a normalization step. It is then assumed in the rest of the code base that rules are in a normalized form. For example, we allow the user to type rules like a(x, y) b(z, w), y = z -> c(x, w) which we then translate to a(x, y) b(y, w) -> c(x, w)

The task is therefore to provide a separate data structure for rules for parsing and planning.

We have now had for some time the split between Rules and ChaseRules, which at least solves the problem outlined in this issue. A major rework of the rule model(s) is still needed, but I would suggest putting up a new issue for that with more concrete demands


Failure to verify on-chain for bb.js-generated ecdsa signature verification

I've spent quite some time tracking this down, but I'm confident that bb.js proofs that include the ecdsa_secp256k1::verify_signature method fail to verify on-chain, but verify off-chain.
Here's an example circuit:
use dep::std;

fn main(pub_x : [u8; 32], pub_y : [u8; 32], signature : [u8; 64], hashed_message : [u8; 32]) {
    assert(true == std::ecdsa_secp256k1::verify_signature(pub_x, pub_y, signature, hashed_message));
}

A fork of noir-starter with a reproducible example is here
As you can check on the test script, off-chain verification works normally (with bb.js) yet the script fails to verify on-chain
The proof verifies both on-chain and off-chain if it's generated with nargo
Tested with versions 0.3.6 and 0.4.2 of bb.js, and versions >0.10.0 of Noir

cc @rahul-kothari

Is this still an issue? Btw bb.js issues should be in the barratenberg repo rather than here

Is this still an issue? Btw bb.js issues should be in the barratenberg repo rather than here (also thanks Savio for cc-ing)

What do you mean with still? I can second that the versions @signorecello has tested with does not work. If it's resolved with a newer version I would love to know :)

@rahul-kothari @signorecello any updates on this?

Noir ≥v0.11.0 now pulls in bb binaries for Solidity verifier generation, which might have resolved the potential cause of this issue (i.e. aztec_backend_barretenberg generating a Solidity verifier with a different verifying key that the version of bb.js used was expecting.)
@FilipHarald if you'd like to give it an early spin, using the latest Nargo + latest noir_js (compile from source) + bb.js v0.7.2 could work. Definitely report back if that's not the case.
@signorecello is updating noir-starter to use noir_js in the meantime. Once that's done he should be happy to check if this issue still persists with the updated noir-starter 🧑💻

Thank you for your reply @Savio-Sou . OK, I think I'll wait for the example in noir-starter.

Reporting that noir-examples is now updated @FilipHarald but this is still an issue unfortunately.
The Noir team will be working on this during the next week or so

Thank you for the update! 🫡


Multiple frameworks doesn't work with VS test explorer

Hi, latest version working fine with one framework with Test Explorer and dotnet test, but if you select more than one framework in project.json working only dotnet test, but not in Test Explorer, all tests disappear.
My project.json:
{
    "version": "<IP_ADDRESS>",
    "dependencies": {
        "MyLib": {
            "target": "project"
        },
        "NUnit": "3.2.1",
        "dotnet-test-nunit": "3.4.0-alpha-2"
    },
    "testRunner": "nunit",
    "frameworks": {
        "net451": { },
        "netcoreapp1.0": {
            "imports": [
                "dnxcore50",
                "netcoreapp1.0",
                "portable-net45+win8"
            ],
            "dependencies": {
                "Microsoft.NETCore.App": {
                    "version": "1.0.0-*",
                    "type": "platform"
                }
            },
            "buildOptions": {
                "define": [ "NETCORE" ]
            }
        }
    }
}

Thanks, I will test this and see if I can figure out what is going on.

@h0useRus I had feedback today that this is a known issue with all test runners in Visual Studio with .NET Core at the moment. I am going to mark this as blocked since it is a Visual Studio issue, not an NUnit issue as far as I know.

@h0useRus I have updated a test repository at https://github.com/rprouse/NUnitWithDotNetCoreRC2 with multiple targets. I am seeing tests in test explorer, but I am only seeing one set of tests and I don't know what framework they target. As I said, that is a known issue.
I do not know why your tests are not appearing. If you can provide more info, I can help you out. You should also try beta 2.

Thank you, great support, will check and wait MS guys to fix VS bugs. As I understand VS run first found framework of tests.

@rprouse Do you have link to ticket/issue for this bug in VS Net Core? To be able to track this error, thanks.

Does it help to reverse the order of the frameworks in the project.json? I.e., have the netcoreapp1.0 one before the net451 one? In Rx.NET, I have both desktop and netcore target frameworks in my test project (with xunit) and it does show up in VS. Running on the command line executes both.

@psulek I do no have the link for the bug in .NET Core. I was talking to the VSTest team last week and to @bradwilson from xUnit about it.

@h0useRus, if you have multiple frameworks in project.json, then the tests shouldn't disappear. Only the predicament is that we have explicit means of seeing which framework is currently selected by VS.
If your project is not public, then looking at your project.json, I can suggest few things:
Use this:
{
    "version": "<IP_ADDRESS>",

    "dependencies": {
        "MyLib": {
            "target": "project"
        },
        "dotnet-test-nunit": "3.4.0-alpha-2"
    },

    "testRunner": "nunit",

    "frameworks": {
        "net451": { },
        "netcoreapp1.0": {
            "imports": "portable-net45+win8",
            "dependencies": {
                "Microsoft.NETCore.App": {
                    "version": "1.0.0-*",
                    "type": "platform"
                }
            }
        }
    }
}

specifying NUnit is not required as it is a sub-dependency of dotnet-test-nunit.
recursive import to netcoreapp1.0 is unnecessary.
dnxcore50 is a past history and deprecated.
the only reason we need "imports": "portable-net45+win8", is because nunit has portable profile (which works fine with netcoreapp on multiple platforms), but not explicitly support .NET Core yet.
if you are using define: NETCORE to be able to have #if NETCORE pre-processor directive in your code, then use #if NETCOREAPP1_0 and no need for extra defines.

After that you can close VS, delete local packages directory (and %USERPROFILE%\.nuget to get a fresh start) and run dotnet restore --no-cache then dotnet test your/test/project/dir. Testing from VS at this point should also work, but it will only show test for one framework (first in the list). If you are still not seeing Tests in Test Explorer with more than one framework, then this is separate issue with Visual Studio. At least in community edition, it is showing the tests even when project.json has more than two frameworks specified.

I've been playing with a multi-framework project today and noticed that 'dotnet-test-nunit' doesn't currently work with classic .NET assemblies. I wonder if this could be related? Here is the repro:
project.json
{
  "version": "1.0.0-*",
  "buildOptions": {
    "emitEntryPoint": true
  },
  "frameworks": {
    "netcoreapp1.0": {
      "imports": "dnxcore50",
      "dependencies": {
        "Microsoft.NETCore.App": {
          "type": "platform",
          "version": "1.0.0"
        }
      }
    }
    ,
    "net451": {
      "buildOptions": {
        "define": [ "NET451" ]
      }
    }
  },
  "dependencies": {
    "NUnit": "3.4.1",
    "dotnet-test-nunit": "3.4.0-beta-1"
  },

  "testRunner": "nunit"
}

test.cs
namespace MultiTarget
{
    using NUnit.Framework;
    using System;

    public class Program
    {
        [Test]
        public static void Main()
        {
            Console.WriteLine("Hello, World!");
        }
    }
}

dotnet run -f net451
Project MultiTarget (.NETFramework,Version=v4.5.1) was previously compiled. Skipping compilation.
Hello, World!

dotnet test -f net451
Project MultiTarget (.NETFramework,Version=v4.5.1) was previously compiled. Skipping compilation.
NUnit .NET Core Runner 3.4.0
Copyright (C) 2016 Charlie Poole
Runtime Environment
    OS Platform: Windows
     OS Version: 10.0.10586
        Runtime: win10-x64
Test Files
    C:\Users\Jamie\Documents\Visual Studio 2015\Projects\MultiTarget\src\MultiTarget\bin\Debug\net451\win7-x64\MultiTarget.exe
System.DllNotFoundException: Unable to load DLL 'Microsoft.DiaSymReader.Native.amd64.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
   at Microsoft.Extensions.Testing.Abstractions.FullPdbReader.CreateSymReader64(Guid& id, Object& symReader)
   at Microsoft.Extensions.Testing.Abstractions.FullPdbReader.CreateNativeSymReader(Stream pdbStream)
   at Microsoft.Extensions.Testing.Abstractions.FullPdbReader..ctor(Stream pdbStream)
   at Microsoft.Extensions.Testing.Abstractions.PdbReaderFactory.Create(String pdbPath)
   at Microsoft.Extensions.Testing.Abstractions.SourceInformationProvider..ctor(String pdbPath, IPdbReaderFactory pdbReaderFactory)
   at NUnit.Runner.Navigation.NavigationDataProvider.GetNavigationDataProvider(String assemblyPath)
   at NUnit.Runner.TestRunner.Execute()
   at NUnit.Runner.TestRunner.Run(String[] args)

I'll investigate and report back.

Huh! I wasn't aware it was supposed to work with classic .NET assemblies.

@jcansdale, while I can't reproduce this locally, #54 fixes this issue which was originally reported in #45.
@rprouse any ETA on next (beta) release? :)

#54 fixes this issue which was originally reported in #45.

Do you know what this fix does exactly? It seems strange for .NET 4.5.1 to be taking an dependency on a .NET Code thing. To be honest I don't understand what "Microsoft.NETCore.Platforms" is!
"net451": {
  "dependencies": {
    "Microsoft.NETCore.Platforms": "1.0.1"
  }
}

Do you know what this fix does exactly?

This is the CLI change; that it requires this package to resolve the dependency graph for transient dependencies etc. The point of #54 is to prevent each consumer of the dotnet-test-nunit package to include this additional dependency, who is targeting net45x TxM.

@jasonwilliams200OK I'm afraid I didn't understand half of that, but I really want to! Could you unpack it a bit for me?

@jcansdale, no problem, this is mostly undocumented. I actually went through couple of related issues in dotnet/cli and xunit repos to understand why dotnet test fails, in preparation for #54. Here is the most relevant bit https://github.com/dotnet/cli/issues/3016#issuecomment-225976094 which points to this blog for explaining the purpose of Microsoft.NETCore.Platform: http://www.natemcmaster.com/blog/2016/05/19/nuget3-rid-graph/.
The choice is to either include Microsoft.NETCore.Platform as dependency or add "runtimes" section with explicit list of RIDs, which is explained in this doc: https://docs.nuget.org/consume/projectjson-format (under Runtimes heading; no internal link on their site..).
The third option is to have a runtime.json file side-by-side our project.json to merely extract out the runtimes section in a separate file.
Since our library has no native-dependency and we want to support widest-possible audience, we want all permutations of platforms x RIDs to be included. To do that, the easiest way is to include Microsoft.NETCore.Platform (which just has a "runtime.json" file listing all the permutations for each supported OS, as we can find in this package's source: https://github.com/dotnet/corefx/blob/master/pkg/Microsoft.NETCore.Platforms/runtime.json).

any update on the next beta release?

I wasn't sure there was enough to warrant another release, but there have been a number of fixes. Let me look at what has been fixed and if there is enough, I will do a release this week.
@deathrowe which fixed issues are you interested in?

@rprouse specifically the fix from https://github.com/nunit/dotnet-test-nunit/issues/45

@deathrowe that's a good one, I'll see what I can do :+1:

@deathrowe, FYI, beta-2 is released: http://www.nuget.org/packages/dotnet-test-nunit/3.4.0-beta-2
@h0useRus, we have updated the usage in README: https://github.com/nunit/dotnet-test-nunit#usage. Can you test if the bug reproduces with beta-2? If you can, please provide an SSCCE as I wasn't even able to reproduce it with beta-1 and VS2015 Update 3 Community Edition.


Created config setting for API URL

Added option to change API url
You can see there are mock and debugging APIs from https://www.shipstation.com/developer-api/#/reference/

Thanks!