Build failure: Model deploy failure on google_cloud_pipeline_components_automl_tabular.ipynb

Investigate if this is a one-off deployment error, or there is something that needs to be fixed in this notebook.
Notebook
Build details
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_19/3131610175.py in <module>
      7 )
      8
----> 9 job.run()
     10
     11 get_ipython().system(' rm tabular_regression_pipeline.json')

~/.local/lib/python3.9/site-packages/google/cloud/aiplatform/base.py in wrapper(*args, **kwargs)
    673                 if self:
    674                     VertexAiResourceNounWithFutureManager.wait(self)
--> 675                 return method(*args, **kwargs)
    676
    677             # callbacks to call within the Future (in same Thread)

~/.local/lib/python3.9/site-packages/google/cloud/aiplatform/pipeline_jobs.py in run(self, service_account, network, sync)
    250         self.submit(service_account=service_account, network=network)
    251
--> 252         self._block_until_complete()
    253
    254     def submit(

~/.local/lib/python3.9/site-packages/google/cloud/aiplatform/pipeline_jobs.py in _block_until_complete(self)
    347         # JOB_STATE_FAILED or JOB_STATE_CANCELLED.
    348         if self._gca_resource.state in _PIPELINE_ERROR_STATES:
--> 349             raise RuntimeError("Job failed with:\n%s" % self._gca_resource.error)
    350         else:
    351             _LOGGER.log_action_completed_against_resource("run", "completed", self)

RuntimeError: Job failed with:
code: 9
message: "The DAG failed because some tasks failed. The failed tasks are: [model-deploy].; Job (project_id = python-docs-samples-tests, job_id = ...) is failed due to the above error.; Failed to handle the job: {project_number = ..., job_id = ...}"

google_cloud_pipeline_components_automl_tabular.ipynb.zip

/gcbrun

fixed


Documentation fixes

Seems like we are pointing to old h2o versions in our readme -

rsparkling will automatically use the latest Sparkling Water based on the major Spark version provided. In this case, the H2O version required for the latest Sparkling Water is <IP_ADDRESS> for Spark 2.0 and <IP_ADDRESS> for Spark 1.6.

The syntax for versions::install("h2o", "<IP_ADDRESS>") seems to be wrong. It should be install.versions("h2o", "<IP_ADDRESS>")
Also found a documentation bug -> if you have installed more spark versions ( such as 2.0.0 and later 2.1.0) and you call sc <- spark_connect(master = "local") it will fail as the older spark version is installed.
We may mention in the documentation that the call should look like sc <- spark_connect(master = "local", version = "2.1.0") in case of multiple spark versions is installed. But just minor stuff, but better writing it down here

Sorry, created the issue twice


Release 1.1.0

Unit test?

Your change was integrated directly in this branch of bh-api-config, removing the entire bundle from the project. Since, it's not used anymore, I suggest to remove the bundle entirely, and to close your PR once the feature is in production


pyarrow.compute.Expression.to_substrait() is missing conversions

Describe the usage question you have. Please include as many useful details as  possible.
I'm not sure why this is missing..
pyarrow.lib.ArrowNotImplementedError: No conversion function exists to convert the Arrow function ends_with to a Substrait call
Functions like pyarrow.compute.ends_with() should be mapped to substrait's ends_with()..
https://substrait.io/extensions/functions_string/
ends_with[¶](https://substrait.io/extensions/functions_string/#ends_with)
Implementations:
ends_with(input, substring, option:case_sensitivity): -> return_type

input: The input string.
substring: The substring to search for.
0. ends_with(varchar<L1>, varchar<L2>, option:case_sensitivity): -> BOOLEAN
1. ends_with(varchar<L1>, string, option:case_sensitivity): -> BOOLEAN
2. ends_with(varchar<L1>, fixedchar<L2>, option:case_sensitivity): -> BOOLEAN
3. ends_with(string, string, option:case_sensitivity): -> BOOLEAN
4. ends_with(string, varchar<L1>, option:case_sensitivity): -> BOOLEAN
5. ends_with(string, fixedchar<L1>, option:case_sensitivity): -> BOOLEAN
6. ends_with(fixedchar<L1>, fixedchar<L2>, option:case_sensitivity): -> BOOLEAN
7. ends_with(fixedchar<L1>, string, option:case_sensitivity): -> BOOLEAN
8. ends_with(fixedchar<L1>, varchar<L2>, option:case_sensitivity): -> BOOLEAN

https://arrow.apache.org/docs/python/generated/pyarrow.compute.ends_with.html
>>> import pyarrow.compute as pc
>>> import pyarrow as pa

>>> exp = pc.ends_with(pc.field("first_name"), "rine")
>>> exp
<pyarrow.compute.Expression ends_with(first_name, {pattern="rine", ignore_case=false})>

>>> substrait_expression = exp.to_substrait(pa.schema([pa.field('first_name', 'string')]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/_compute.pyx", line 2458, in pyarrow._compute.Expression.to_substrait
  File "pyarrow/_substrait.pyx", line 247, in pyarrow._substrait.serialize_expressions
  File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: No conversion function exists to convert the Arrow function ends_with to a Substrait call
>>>

Component(s)
C++, Python

I've also observed similar but would like to understand how it's possible to add these, can be they be configured via an API or does it require change to the underlying code.l


Instance creation fails on macOS with recent Vulkan SDK

Running on MacOS with Vulkan SDK 1.3.231.
The Manager::createInstance() fails silently when calling
vk::createInstance(&computeInstanceCreateInfo, nullptr, this->mInstance.get());

Perhaps it should check the returned code, which in this case happens to be VK_ERROR_INCOMPATIBLE_DRIVER.
It seems that the recent SDK requires the VK_KHR_PORTABILITY_ENUMERATION_EXTENSION_NAME extension to be enabled on MacOS:
applicationExtensions.push_back(VK_KHR_PORTABILITY_ENUMERATION_EXTENSION_NAME);
computeInstanceCreateInfo.flags |= vk::InstanceCreateFlagBits::eEnumeratePortabilityKHR;
...
vk::createInstance(&computeInstanceCreateInfo, nullptr, this->mInstance.get());

@Archie3d it seems it took a while to get back to this but we have finally been able to run Kompute reliably on an M1 device.
Specifically there are a few ways in which this can be made work, one option is to install an older SDK - this has been tested correctly with driver 1.2.198, as long as the VK_ICD_FILENAMES env var is set up correctly to your MoltenVK driver (e.g. $VULKAN_SDK/share/vulkan/icd.d/MoltenVK_icd.json)
Furthermore we should certainly extend the code to include a check for compatibility, and enable users to add the KHR_PORTABILITY_ENUMERATION_EXTENSION_NAME as suggested.


lsp: Update LSP linting to run incrementally after file change

Fixes to https://github.com/StyraInc/regal/issues/1133
This PR is a large refactoring of the language server diagnostics loops. Changes:

when the contents of a file with aggregate violations changes, this no longer triggers a full workspace lint
when the contents of any file changes, the aggregate rule state is now cached on the server and the results of this update, in addition to the full global state, is used to perform a linting run of all files but only linting for aggregate rules.
new tests have been added to cover this functionality and the underlying functionality which these server changes depend on in the linter.
server tests moved to new files since they are very long.
increased timeouts for go test when running in go test -race https://go.dev/doc/articles/race_detector#Runtime_Overheads
updated the cache to allow partial updating of diagnostics based on the relevant rules.

I've tried this now and it's a huge improvement 👏 No more lag or even freezes, but most things just show up right away, and the aggregate violations a little later. This is great!

aggregate violations a little later

Yeah, I actually think that we are in a position to correct that, but I'd need to do some more testing. This is because for a per-file run, only the non agg rules are run to improve performance. Since we know that a full workspace agg rule run will take place immediately after with the updated aggregate state.
I think the solution is to provide all aggregate data but for the current file, and then run all rules in the per file lint run. It should be simple enough, but something I think might approached in another PR....


Fix #190: Validation register

Added register screen validation with supporting text. I did not think it was a good practice or not here, just wanted to try. Feel free to share your opinion. I didn't write those test cases. I'll write them again if this was merged. Thank you.

I think you could simply add a new check among other checks at LoginService.kt or at RegisterService.kt

I think you could simply add a new check among other checks at LoginService.kt or at RegisterService.kt

I also thought that, just wanted to try and tried. I'll commit them by doing that.

I may close this PR as of now. Thank you and sorry 😅

No problem, I'll create another one


EDITOR-#467 PositionMap, PositionFrame, RequestEdit resolvers

What is this PR for?
Complete all PositionMap's, PositionFrame's, RequestEdit resolvers and some data types.
What type of PR is it?
[Feature]
Todos
None
What is the Github issue?
close #467
How should this be tested?
graphql playground
Screenshots (if appropriate)
Questions

I have use graphql playground and mysql in terminal to check the code.

Check if the generated schema matches the old one, for example, you need to rename "posMap" to "PosMap" by "#[graphql(name = "PosMap")]"
PosMap is a map of id and frame
If you only need certain fields of data, select them in SQL query, not filter them by "map"

Check if the generated schema matches the old one, for example, you need to rename "posMap" to "PosMap" by "#[graphql(name = "PosMap")]" PosMap is a map of id and frame If you only need certain fields of data, select them in SQL query, not filter them by "map"


CORE-9975: Allow sign operation to take a key category in context argument

In particular, now that the API is frozen I think a cateogry key in the context Map<String,String> parameter  is the best way to communicate a category.


Use markdown frontmatter to provide Table of contents, language and frontmatter rendering

Add control for the rendering of the frontmatter
Add control to include a TOC
Add control to set language - allows control of ToC header and CJK glyph choice.

Signed-off-by: Andrew Thornton<EMAIL_ADDRESS>

Codecov Report

Merging #11047 into master will decrease coverage by 0.14%.
The diff coverage is 9.14%.

@@            Coverage Diff             @@
##           master   #11047      +/-   ##
==========================================
- Coverage   43.47%   43.33%   -0.15%
==========================================
  Files         600      603       +3
  Lines       85108    85416     +308
==========================================
+ Hits        37002    37016      +14
- Misses      43537    43826     +289
- Partials     4569     4574       +5

Impacted Files
Coverage Δ

modules/markup/html.go
77.38% <0.00%> (-2.22%)
:arrow_down:

modules/markup/markdown/ast.go
0.00% <0.00%> (ø)

modules/markup/markdown/renderconfig.go
0.00% <0.00%> (ø)

modules/markup/markdown/toc.go
0.00% <0.00%> (ø)

modules/markup/markdown/goldmark.go
41.66% <19.82%> (-23.30%)
:arrow_down:

modules/markup/markdown/markdown.go
90.90% <100.00%> (ø)

modules/markup/sanitizer.go
92.77% <100.00%> (+0.27%)
:arrow_up:

models/unit.go
41.97% <0.00%> (-2.47%)
:arrow_down:

modules/log/event.go
64.61% <0.00%> (-1.03%)
:arrow_down:

routers/repo/view.go
36.14% <0.00%> (-0.68%)
:arrow_down:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9588d2c...3ebd430. Read the comment docs.

make lg-tm work

make lg-tm work


Harvesting bugged in 2.4.2

Describe the bug
Colonists do not cut fully grown plants, unless they're manually given the harvest order.
Steps to reproduce the behavior (VERY IMPORTANT)

Create new save (vanilla)
Designate a grow zone
Wait for it to be fully harvestable, make sure a colonist has plant cutting job assigned to them
See error

Error Log
https://p.teknik.io/hqPw9
Save file
harvest bug.rws
Screenshots

Mod list (For vanilla report, it has to be same as below list.)

Harmony
Core
Royalty
RimThreaded

I was not able to reproduce the error after several tries, however i can confirm that the issue occurs.

To add additional information, it seems that having a colonist prioritize cutting is not the only way, you can also mark the plant for harvest by pressing H and it'll be harvested

Hi, after testing the new patch, it seems it fixes harvesting, but breaks colonists who try to harvest. It outputs these errors for a while

Afterwards, colonists behavior seems to be broken somewhat. Instead of wandering, they just stand in one place until they have some work to do, and they don't seem to do all work (?) like one colonist would not sow plants until I manually had them do it.
This is the log file https://gist.github.com/HugsLibRecordKeeper/686fa589155bfbb7fd0938bfa8d80d56

I have to tell them to cut the plants. Then they rush to do it. On their own, they will literally "Wander" around before they harvest anything, even when locked in the grow zone.


Update about text

[ ] Add language that this interface is just a demonstration of some of ODIS' capabilities.
[ ] Explain / link to documentation which describes that users can leverage the same architecture and code to build their own instances of a search (regional portals, etc)
[ ] Add disclaimer stating that OIH does not curate or ensure the correctness of partner (meta)data.

@pbuttigieg where do we place this text? In the search interface itself or on a separate page?

Separate page that will be the target of an "About" link in a header or hamburger menu


Can't use AModule::AConst = 1 to declare a constant

ModuleFoo::Bar = 1

{% puts ModuleFoo.class_name %} # => NumberLiteral

puts ModuleFoo::Bar  # undefined constant ModuleFoo::Bar

https://carc.in/#/r/3mjb
I would expect it to create a module named ModuleFoo with a constant Bar inside it.
Instead, ModuleFoo::Bar = 1 creates a constant ModuleFoo with value 1, discarding Bar completely.
Apart from the fact that the current behavior is a bug, as we already have a few other places where we can use a module name in front of something, I think it would be consistent to allow this for constants too.

Yeah the current behaviour is a bug but I'd also be fine with this being a compile error.


Sending data through HTTP server

Checking some data I noticed that the integration version for the data sent through the HTTPserver is “api”. I believe that somehow it was the intended behaviour, since when creating the definition for the HTTP server the agent does not know where the payload is coming for and therefore set “api”.
When emitting V3 data the agent is taking the integration version from the payload and the integrationName from the definition.

A possible solution at agent lever would be to take that as well the integrationName from the payload at least when it is set to a generic “api”

https://github.com/newrelic/infrastructure-agent/issues/879
new agent release will fix this


SubscriberClient.testIamPermissions invalid return type

https://github.com/googleapis/nodejs-pubsub/blob/db1f69c4aac5b3abfea12f70d45ceb8b7c957388/src/v1/subscriber_client.ts#L2186
It seems that there is an error in SubsriberClient.testIamPermissions method return type. It says that the return type is IamProtos.google.iam.v1.TestIamPermissionsResponse but the method returns an array where the first element is IamProtos.google.iam.v1.TestIamPermissionsResponse.

Moving this to a backlog cleanup item, because it's probably going to involve some breaking changes in gax, which the whole googleapis world relies on. :|


Make a dashboard example

Mature some of the github widget examples into a dashboard view of some data.

Nice! @bollwyvl


Replace makefile with deno.jsonc

Makefiles are so last week (also it would act as an additional example of how to use the new task runner).

Makefiles are so last week

nope, not sure why you say that.

it would act as an additional example of how to use the new task runner

yeah, good point and we should do deno task for this reason.

@littledivy

Makefiles are so last week
nope, not sure why you say that.

It was both a light-hearted joke and a celebration of Deno tasks which had been introduced in the previous week. Thanks for resolving this!


on desktop TextField lacks a lot of native keyboard reactions

Create a (multiline) TextField in a desktop application. Try the following hotkeys:

arrow up and arrow down (expected that caret moves up and down - however nothing happens in reality).
SHIFT + cursor arrows (expected to create selection - nothing happens)
CMD+A (expected to select all content - nothing happens)
select some text press DELETE (expected to delete selected text - nothing happens)
select text starting from the first character and press BACKSPACE (expected to delete selection - however if the first character is selected then nothing happens, works if selection does not start at the very first character).

Duplicate of https://github.com/flutter/flutter/issues/30725


Infinite loop / pathalogical runtime

Hello!
I've been having this problem since the 0.28 re-write of the union / intersection code, I've been sticking to 0.27 for now - but I spent a bunch of time today trying to narrow down a reproducible case I can easily share.
When I run flow on my whole project, I get this in the output after just under 3 minutes real-time:
parsed 25045 files (2322 ok, 22723 skipped, 0 failed, 67 resource files) in 2.379454
TimingEvent `Parsing`: start_wall_age: 0.014958; wall_duration: 2.379646
TimingEvent `PackageHeap`: start_wall_age: 2.394628; wall_duration: 0.215722
TimingEvent `InitLibs`: start_wall_age: 2.610385; wall_duration: 0.332659
inferred 2322 files in 1.909123
TimingEvent `Infer`: start_wall_age: 2.943086; wall_duration: 1.909220
TimingEvent `CommitModules`: start_wall_age: 4.930310; wall_duration: 0.658399
TimingEvent `MakeMergeInput`: start_wall_age: 5.588743; wall_duration: 0.000000
TimingEvent `ResolveDirectDeps`: start_wall_age: 5.588749; wall_duration: 0.000002
TimingEvent `CalcDeps`: start_wall_age: 5.588765; wall_duration: 0.012299
Worker interrupted with signal: sigill
Failure("Subprocess(60528): fail 2")

During execution a single flow process sits at 100% and continually increases in RAM usage (into GBs) - the process ID listed in the error output doesn't match the one which pegged a core.
Through process of elimination, I've identified a single file which when excluded from flow cause the total check to only take about 8 seconds. This file is my type definition for immutable records, which are then used heavily throughout the rest of the application. This is what it looks like at the moment: https://gist.github.com/glenjamin/75a96b45f4bb5c6ac221815d28c548dd/7d599d2b6d5bc038f2ba120c448a1c34763398b8#file-records-js
I'm still working on trimming this down to a smaller repro case, I'll update this post when I have more.

I've managed to reduce this down pretty small through lots of not-fun trial and error: https://github.com/glenjamin/flow-stuck-repro/tree/d01ba7ecdeec27c35565ff2666ed2afc6fe6ba34
I think it'll probably reduce a bit further, so I'll have another crack at it later.
As it does seem to be this particular function, I expect I'll be able to write the same thing in a different way and work around this bug in my actual app.

Thank you for digging in here! The work you've done will make this a lot easier to track down and fix.

I've managed to distill things down a bit further.
https://github.com/glenjamin/flow-stuck-repro/tree/30e8dbe993a27bff507cda176fb7a5750461c737
Weirdly, it seems to complete type checking and the processes that breaks doesn't OOM and actually terminates in a semi-reasonable timeframe - but it still produces all of the symptoms of the larger case.

Any other findings from this? One of our engineers just ran into this issue as well and hoping there's some knowledge to be gained from this issue to point him in the right direction.

I have the same issue. Anyone managed to solve this?

Yes I also have the same kind of problem ! When using record modeling like in this issue : #2062

I don't get much in my logs at all:
[2016-12-28 12:39:44] Initializing Server (This might take some time)
[2016-12-28 12:39:44] executable=/Users/Administrator/projects/myproject/node_modules/flow-bin/flow-osx-v0.37.4/flow
[2016-12-28 12:39:44] version=0.37.4
[2016-12-28 12:39:44] Parsing
Worker interrupted with signal: sigill
End_of_file

I don't think it's the fault of the codebase, since flow works correctly for the same project on my coworker's macs.
I'm on macOS Sierra 10.12.
Is there any flags I could pass to flow to make it print more information into logs to pinpoint the problem?

Same here:
[2017-01-20 16:20:22] Initializing Server (This might take some time)
[2017-01-20 16:20:22] executable=/Users/hmeerlo/Dev/jbkantoor/node_modules/flow-bin/flow-osx-v0.38.0/flow
[2017-01-20 16:20:22] version=0.38.0
[2017-01-20 16:20:22] Parsing
[2017-01-20 16:20:30] Building package heap
[2017-01-20 16:20:31] Loading libraries
[2017-01-20 16:20:33] Running local inference
[2017-01-20 16:20:39] Re-resolving directly dependent files
[2017-01-20 16:20:39] Calculating dependencies
[2017-01-20 16:20:39] Merging
Worker interrupted with signal: sigill
End_of_file
[2017-01-20 16:20:48] Server is READY
[2017-01-20 16:20:48] Took 25.802131 seconds to initialize.
[2017-01-20 16:20:48] Status: OK

I built flow from source and ran it with --max-workers 0 to be able to debug it with lldb. This crashed with a EXC_BAD_ACCESS. But then I built it using target 'build-flow-debug' form the Makefile. Interestingly this produced the following error:
Process 3122 launched: '../flow/bin/flow' (x86_64)
(3122) merge_strict_job THROWS: [1] /Users/hmeerlo/Dev/jbkantoor/src/components/Planning/Planning.js: Stack overflow

src/components/Planning/Planning.js:1
  1: /* @flow */
     ^ merge_strict_job exception: Stack overflow

....

Found 17 errors
Process 3122 exited with status = 2 (0x00000002)

Any idea what has caused this stack overflow?

I tried going back in git history to the very beginning of my project, when flow definitely worked before, but it still crashes.
I tried removing node_modules and removing everything from .flowconfig, no result.
It now doesn't work on any of my coworkers macs either.
I tried increasing allowed number of file handles in macOS, and increasing shared memory through flow flags, and it still made no difference.
I also tried it on Windows and it works just fine without any problems. To me it sounds like it's something to do with macOS

I agree that it must be something macOS specific (or the macOS OCAML compiler specific). I find it strange that I get 3 different errors:

SIGILL on the 0.38.0 release used by flow-bin
SIGSEGV -> EXC_BAD_ACCESS on self built 0.38.0 release using max-workers 0
Stack Overflow on debug build of 0.38.0

It might be that the SIGILL issue is the same as the stack overflow issue but that the stack overflow is only detected in debug builds and in production builds it just lets the stack overflow resulting in illegal instructions (SIGILL)?
I never did anything with OCAML before so it is hard for me to debug this, but it is a significant issue because it makes flow very unreliable for me.

That sounds valid. The actual error is stack overflow, which results in SIGIL in an optimised release build.

Ok, I can not let this go by. I appreciate flow too much :-) So I learned the basic syntax of OCaml and managed to print the stack trace of the recursion that has caused this. Unfortunately my OCaml knowledge is not good enough yet to solve the issue:
Called from file "map.ml", line 269, characters 25-40
Called from file "src/typing/flow_js.ml", line 8516, characters 4-467
Called from file "src/typing/flow_js.ml", line 8373, characters 4-29
Called from file "src/typing/flow_js.ml", line 8532, characters 4-54
Called from file "src/typing/flow_js.ml", line 8520, characters 10-62
Called from file "map.ml", line 269, characters 44-63
Called from file "map.ml", line 269, characters 25-40
Called from file "src/typing/flow_js.ml", line 8516, characters 4-467
Called from file "src/typing/flow_js.ml", line 8373, characters 4-29
Called from file "src/typing/flow_js.ml", line 8532, characters 4-54
Called from file "src/typing/flow_js.ml", line 8520, characters 10-62
Called from file "map.ml", line 269, characters 44-63
Called from file "map.ml", line 269, characters 25-40
Called from file "src/typing/flow_js.ml", line 8516, characters 4-467
Called from file "src/typing/flow_js.ml", line 8373, characters 4-29
Called from file "src/typing/flow_js.ml", line 8532, characters 4-54
Called from file "src/typing/flow_js.ml", line 8520, characters 10-62
Called from file "map.ml", line 269, characters 44-63
Called from file "map.ml", line 269, characters 25-40
Called from file "src/typing/flow_js.ml", line 8516, characters 4-467
Called from file "src/typing/flow_js.ml", line 8373, characters 4-29
Called from file "src/typing/flow_js.ml", line 8532, characters 4-54
Called from file "src/typing/flow_js.ml", line 8520, characters 10-62
Called from file "map.ml", line 269, characters 44-63

--- recurring hundreds of times

@hmeerlo those line numbers don't match up with 0.38 or master anymore. you said you compiled it yourself, right? which commit are you on?

Ah, checked out this afternoon, already 3 commits behind: 46fffaccd545664ec6ff1b86d5fe32a8e9a1f387

Line numbers for current master:
Called from file "map.ml", line 269, characters 44-63
Called from file "map.ml", line 269, characters 25-40
Called from file "src/typing/flow_js.ml", line 8524, characters 4-467
Called from file "src/typing/flow_js.ml", line 8381, characters 4-29
Called from file "src/typing/flow_js.ml", line 8540, characters 4-54
Called from file "src/typing/flow_js.ml", line 8528, characters 10-62
Called from file "map.ml", line 269, characters 44-63
Called from file "map.ml", line 269, characters 25-40
Called from file "src/typing/flow_js.ml", line 8524, characters 4-467
Called from file "src/typing/flow_js.ml", line 8381, characters 4-29
Called from file "src/typing/flow_js.ml", line 8540, characters 4-54
Called from file "src/typing/flow_js.ml", line 8528, characters 10-62
Called from file "map.ml", line 269, characters 44-63

@glenjamin I tried your repro repo in 0.38 and now get
operations.js:0
inconsistent use of library definitions
module. *** Recursion limit exceeded ***
module

@hmeerlo you still get a SIGILL on 0.38/master so I think these are possibly different underlying issues. maybe.

I distilled @glenjamin's repro down further: https://flowtype.org/try/#0PQKgBAAgZgNg9gdzCYAoVATApgYxgQwCcsw98BncsAISIB4BpAPjAG9UwwBXABw3wAuWRgH0mACl78hhAFxhxAN3wwuWeQIAWAS3IBKMAF4WtQqKZ75p8wG5UAX3TYyxMFC4A7HAO1wPbuDg6AFUJDjBtD20fFQA1FTV5YIAacJwVGAAjfBwAaygPeXEeYkVfLnJ41XUwYINjWtRLWrtMXAJXZUIIoQBbcit6cgFCSIBzJjtnDpIu7j5BLDkFZWrBs2HRjwn6kyGR8cn0KEDxcOisftTOcRycXbA7gDopRckFmT0muyA

FYI: it still happens in 0.40.0

any progress on this one?
is there any workaround?


GitHub fetcher

Implement CommitData pojo, that contains all the fields from the received JSON, required for the project.
Implement fetcher function to receive the JSON from the GitHub API (using Webhooks).

I'd also really recommend putting some extra time into structuring the description of your Pull Requests. Although there aren't necessarily any visual elements here, The description makes references to context that isn't readily available for the reviewer (Received JSON, the project, GitHub WebHooks, POJO).


Unknown IP Address on gateway connected to

I am seeing the gateway connected to this ip address - <IP_ADDRESS>. But I counld not find any information about this ip. I am guessing polygon lap. But I have not been making any request related to polygon network.
So anyone here can help to me understand why the gateway need to connect to this ip.
thanks
Tam

Hi @delgemoon thanks for reporting however i dont recall pointing to specific ip address for gateway specially the one provided on this ticket. Also we would appreciate if you would follow the template upon opening the bug report else we can't reproduce the issue reported here.
If you still need us to investigate, kindly reopen with supporting files and steps to reproduce. Thank you


AzureWebJobsSecretStorageType must be set to Files

I got an error when creating the skillset.
Debugging revealed some API issue -- got fixed by setting the AzureWebJobsSecretStorageType to Files in the App settings of the App Function.

Added this setting to the ARM template so that it will be there automatically from now on, thanks!


A couple of changes in bpf/nat to help adding new support of ICMP types

Refactoring some part of the code in lib/nat.h to make easier future support of ICMP type messages

BPF check test failing because of #22043. I will rebase after the change is merged.

o/ @joestringer just in case you have not noticed it. I have addressed you request to separate some commits from #18414

/test

/test

@pchaigno do you see why the CI is failing? I'm not able to understand the reason when looking into Jenkins

/test-1.16-4.9

Thank you for the quick merge Tobias.


⚠ Bofrost.es has degraded performance

In 7bf83e3, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1940 ms

Resolved: Bofrost.es performance has improved in 25ea016 after 11 minutes.


Remove ban of EXTCODESIZE

Starting the move tentatively decided on during EOF implementers call.
Will proceed to EIPs/EEST/evmone next.
EXTCODEHASH unbanning was also discussed, but the decision was to start small and possibly expand in the next step.

Main argument against would be: this breaks code unobservability guarantees.
In particular the danger can be in some contracts implementing logic conditional on code size (if EXCTODESIZE(addr)=...) and then this code being broken in case addr code is translated to a different version. (here in particular, if legacy code is ever translated to anything else)
And with EIP-7702 even legacy translation is not neeed: result of EXTCODESIZE changes when EOA is delegated to a contract or when delegation is updated. From the point of view of 7702 this is desired behavior.
All other considered solutions (EXT*CALL status codes, IS_CONTRACT etc.) introduce some level of observability, too (some logic depending on whether the contract is EOA or not may break if code assumes this is immutable). But EXTCODESIZE method raises this level up a notch, with more potential scenarios of breakage.

Also general aesthetics / elegance argument: having to explain why with all the measures to prevent code observability. there was one exception for EXTCODESIZE. And why EOF can read legacy code size but not EOF code size.

Agreed to both comments, The only reason why this has been (tentatively) chosen is that it doesn't introduce yet another opcode.
I only have doubts whether this

some contracts implementing logic conditional on code size (if EXCTODESIZE(addr)=...)

pattern (with exact equality, and to a RHS not being zero) is worthwhile. To me the elegance and abstraction-based arguments are much more important.
If we can at this point have an IS_CONTRACT opcode, this would be definitely cleaner.

Per last EOF implementers call this is unlikely to happen, the EXTCODESIZE remaining banned has momentum. Will reopen if anything changes.


Partially infer conv return types

Fixes #47057.
af9ad9d introduces conv inferReturnTypes, but skips the inference when either input or filter does not have static shape (unranked or not all dimensions are static). However, it introduces the regression when it comes to partially dynamic input. This PR instead tries to infer as many dimensions as possible, which avoids shape information loss when there are only some dynamic dimensions (like batch dim).

@smit-hinsu Do you want to take a look since you approved #44022 ?


Can't run static-compress

Hello,
I successfully installed static-compress on my Manjaro with cargo.
However I have an error message when running static-compress :
static-compress : commande introuvable (command not found`)
What could be wrong?
Thank you.

WebP supports requires the cwebp command to be installed.


[WFCORE-4335] null file is created in bin directory when starting sta…

…ndalone.bat
Issue: https://issues.jboss.org/browse/WFCORE-4335

Can one of the admins verify this patch?

this is ok to test


The folder Sales/Transformers don't publish to the project folder

Terms

[X] Before you openning this issue, i have checked if the issue has already been reported.

Bagisto Version(s) affected
2.1.2
Issue Description
Published bagisto project to the folder didn't copy the folder Sales/Transformers
Preconditions
Nothing
Steps to reproduce
composer create-project bagisto/bagisto OR composer create-project bagisto/bagisto website.com
Expected Result
Sales/Transformers
Actual Result
No folder and I have errors with Stripe
After manually copying folder I have another error
Screenshots

Additional context
No response

Please avoid raising duplicate issues. I've already replied in #9829.

It's not duplicate because you sent me to another github in this ticket but the problem in Bagisto core

You are using version 2.1.2, but Sales/Transformers will be introduced in the upcoming release. This issue is a duplicate because you were using a third-party Stripe package that only supports the master branch, not the latest stable release.


MessageTags on Navigation bar

Hello,
I have been facing issue with AppInbox screen.
When I add message tags to the the style of CleverTapInboxViewController, the CleverTap library  adds a segmented control to navigation bar. If I push a new controller from AppInbox screen when a user clicks on notification item, the segmented view doesn't get removed and it has to be explicitly removed.
As soon as a new controller is pushed from AppInbox screen or if its dismissed, the segmented view should be implicitly removed as it is implicitly added by the CleverTap SDK.
This is the code from library which adds the view:
https://github.com/CleverTap/clevertap-ios-sdk/blob/f50fc19b8de30222f66a5cc708d87df72f2dc282/CleverTapSDK/Inbox/controllers/CleverTapInboxViewController.m#L224
This is my code to show AppInbox:
 if let inboxController = CleverTap.sharedInstance()?.newInboxViewController(with: style, andDelegate: self) {
            inboxController.hidesBottomBarWhenPushed = true
            self.navigationController?.pushViewController(inboxController, animated: true)
        }

Thank you!

Hello @Aditi3,
Any update regarding this issue?
Thank you!

Hi @Hamna21
Apologies for the delay in getting back to you.
I see you're implementing pushViewController:animated:. We recommend presenting a CleverTap App Inbox view controller modally.
Nonetheless, we've noted this behavior and the code suggested by you related to the segment control. I request you to allow us some time to debug more here and we will get back with an update shortly.
Thanks,
Aditi

Thank you for your reply @Aditi3.
Well I have to open a deeplink (push a new view controller) when a user clicks on a notification. In case of presenting the CleverTap App Inbox view controller modally, the UX is not very nice. There should be an option of using both cases.
Waiting for more updates on this. Thanks!

Hi @Hamna21
Sorry for not responding earlier! We have been trying to reproduce the scenario explained in your previous comments and have been able to replicate the same at our end. I have added a solution to handle the same in our development branch. It will be included in our next release. You can expect the release by end of this month as the fix comes with a new feature.
For now, and considering you can't implement the recommended way of presenting the CleverTap App Inbox, the quick workaround would be to edit CleverTapInboxViewController class in your Project and add the code to remove segmentedControlContainer from the Superview.
Helping code snippet:
- (void)viewWillDisappear:(BOOL)animated {
    [super viewWillDisappear:YES];
    if (self.segmentedControlContainer) {
        [self.segmentedControlContainer removeFromSuperview];
    }
}

Thank you for notifying.

Hello @Aditi3,
Thank you for the update. Waiting for the next release 👍

Hi @Hamna21
We have released the CleverTap iOS SDK that comes with this fix.
Release link: https://github.com/CleverTap/clevertap-ios-sdk/releases/tag/3.8.0
Thank you again for bringing this up!


Trying to define kl divergence Layer Issues.

Hello!
I am new to Theano and Keras so maybe I am asking something silly but have been a few days trying to get this working and cannot.
I want to build a siamese architecture that takes 3 vectors as inputs and 1 output weather they come from the same source or not. The code I have is something like this:
Here is my custom Lambda layer:
''' custom loss function'''
def kl_distance(vects):
    qry_vec, doc_vec = vects
    qry_vec = K.clip(qry_vec, K.epsilon(), 1)
    doc_vec = K.clip(doc_vec, K.epsilon(), 1)
    return K.sum(qry_vec * K.log(doc_vec), axis=-1)

def relative_distance(vects):
    x, y = vects
    return x - y

def kl_shape(shapes):
    shape1, shape2 = shapes
    return (shape1[0], 1)

def rel_shape(shapes):
    shape1, shape2 = shapes
    return (shape1[0], 1)

Here is my based network
emb_layer = Dense(embedding_dimensionality, activation="linear", name="emb_layer")
rep_layer = Dense(vocabulary_size, activation="softmax", name="rep_layer")

qry = Input(shape=(vocabulary_size,), name="qry_input")
doc_plus = Input(shape=(vocabulary_size,), name="doc_plus_input")
doc_minus = Input(shape=(vocabulary_size,), name="doc_minus_input")

qry_emb = emb_layer(qry)
doc_emb_plus = emb_layer(doc_plus)
doc_emb_minus = emb_layer(doc_minus)

qry_rep = rep_layer(qry_emb)
doc_rep_plus = rep_layer(doc_emb_plus)
doc_rep_minus = rep_layer(doc_emb_minus)

plus_score = Lambda(kl_distance, output_shape=kl_shape, name="qry_plus")([qry_rep, doc_rep_plus])
minus_score = Lambda(kl_distance, output_shape=kl_shape, name="qry_minus")([qry_rep, doc_rep_minus])

predicts = Lambda(relative_distance, output_shape=rel_shape)([plus_score, minus_score])

model = Model(inputs=[qry, doc_plus, doc_minus], outputs=predicts)
model.summary()

Error message:
ValueError: Input dimension mis-match. (input[1].shape[1] = 1, input[2].shape[1] = 16)
Apply node that caused the error: Elemwise{Composite{(i0 - (i1 * i2))}}(TensorConstant{(1, 1) of 1.0}, Elemwise{sgn,no_inplace}.0, Elemwise{Sub}[(0, 0)].0)
Toposort index: 61
Inputs types: [TensorType(float32, (True, True)), TensorType(float32, matrix), TensorType(float32, row)]
Inputs shapes: [(1, 1), (16, 1), (1, 16)]
Inputs strides: [(4, 4), (4, 4), (64, 4)]
Inputs values: [array([[ 1.]], dtype=float32), 'not shown', 'not shown']
Outputs clients: [[Elemwise{maximum,no_inplace}(TensorConstant{(1, 1) of 0.0}, Elemwise{Composite{(i0 - (i1 * i2))}}.0), Elemwise{Composite{(EQ(i0, i1) * i2 * i3)}}[(0, 0)](Elemwise{maximum,no_inplace}.0, Elemwise{Composite{(i0 - (i1 * i2))}}.0, Elemwise{Composite{((i0 * i1) / i2)}}.0, Elemwise{sgn,no_inplace}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

Hello,
This is probably a broadcasting issue relative to theano.
Your input of shape (16,1) is of type TensorType(float32, matrix) which is not broadcastable.
You will have to call a theano.tensor.addbroadcast( theproblematicsymbol )
See documentation there : (http://deeplearning.net/software/theano/library/tensor/basic.html)

Hello,
Thank you for your advice.
In the model input, I set the following:
Here is the code:
model.fit([X_train, X_train, X_train], Y_train,
			batch_size=batch_size,
			epochs=epochs,
			verbose=1,
			shuffle=True)

where
X_train.shape = (800, 51253)
Y_train.shape = (800, )
But input shape is the same.
This is the issue of broadcast?
I am not clear on it . How can I correct it?


13.4 not mentioned in CHANGELOG.md

Could you mention what the changes are please?

thanks we will update it.
nothing breaking, mainly fix to https://github.com/rackt/react-router/issues/2128

Thanks.

thanks @mjackson


Issue with certain spritefonts.

I have a few spritefonts which I am using with Scribble. One of my spritefonts does not display correctly. Some of the characters are repeated at the exclusion of others. See this screenshot, where the top one is incorrect, but the second one is fine.

All the spritefonts are set up the same way. I checked that the collision mask was set up according to the documentation, and I tried putting the spritefont on its own texture page. There is one pixel of border around the font. Perhaps this is user error, but I am unable to solve this issue on my own at this point.
This particular issue is Scribble 6.0.10 for 2.3, which I updated to use today. The issue does not occur with Scribble 5.5.0 for 2.2 as far as I am aware, as it has been working for me previously.
Code:
//Create Event
scribble_init("", "fnt_pixelfont", false);
scribble_add_spritefont("fnt_pixelfont", "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz.,?!'()<>^*+=-/%:;_$", 1, 4);
scribble_add_spritefont("fnt_smallfont", "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz.,()'?!-/$", 1, 3);

//Draw Event
scribble_draw(x, y, "0123456789\nABCDEFGHIJKLMNOPQRSTUVWXYZ\nabcdefghijklmnopqrstuvwxyz\n.,?!'()<>^*+=-/%:;_$");
scribble_draw(x, y + 56, "[fnt_smallfont]0123456789\nABCDEFGHIJKLMNOPQRSTUVWXYZ\nabcdefghijklmnopqrstuvwxyz\n.,()'?!-/$[/f]");

Attached is a testing project which should demonstrate the issue. IDE v<IP_ADDRESS>9
ScribbleTesting.zip

I came here to inquire about this exact issue. Glad it's not just me - I've been trying to figure out for a couple hours where I could've gone wrong! Scribble was working wonderfully when I was using GMS2.2, but today when I updated to 2.3 something seems to have gone awry.
Here's my spritefont in the image editor:

And here's the relevant code in my dialogue object (based on Juju's example project):

But here's how it's showing up when I compile the game:

I've tried deleting and reloading the sprite, rearranging the glyphs in the font string, etc. FWIW the built-in draw_text functions are working fine.

Relates to #111. This bug is caused by some changes in the way GMS2.3.0 handles sprite collisions.

Fixed in 6.0.11


99 object-oriented bottles of beer in Crystal

What does the program do ?
99 bottles of beer
In what programming language it is written?
Crystal

Thank you for participating  in hacktoberfest ❣ if you liked working on this repo you can share it with friends and star this project for helping others


CookieRequestCultureProvider slow performance

I have detected that if the user select a language and I use the CookieRequestCultureProvider, the site is slower.
My code:
RequestCultureProviders = new List<IRequestCultureProvider>
                {
                    new CookieRequestCultureProvider
                    {
                        CookieName = "_cultureLocalization"
                    },
                    new QueryStringRequestCultureProvider(),
                    new AcceptLanguageHeaderRequestCultureProvider
                    {

                    }

                }

If I dont set the language and take default, all is very fast, but if I set and save in cookie, the performance is very slow...
Any ideas? #

Why you need to initialize the RequestCultureProviders while it's already have the default providers?!!
Can you provide a link to your sample for further investigation

HI @hishamco Here you have the rest of the code involved in Localization:
var requestLocalizationOptions = new RequestLocalizationOptions
            {
                // Set options here to change middleware behavior
                SupportedCultures = new List<CultureInfo>
                {
                    new CultureInfo("en-US"),
                     new CultureInfo("fr-FR"),
                     new CultureInfo("pt-PT"),
                     new CultureInfo("pt-BR"),
                     new CultureInfo("it-IT"),
                     new CultureInfo("de-DE"),
                    new CultureInfo("es-ES")
                },
                SupportedUICultures = new List<CultureInfo>
                {
                     new CultureInfo("en-US"),
                     new CultureInfo("fr-FR"),
                     new CultureInfo("pt-PT"),
                     new CultureInfo("pt-BR"),
                     new CultureInfo("it-IT"),
                     new CultureInfo("de-DE"),
                     new CultureInfo("es-ES")

                },
                RequestCultureProviders = new List<IRequestCultureProvider>
                {
                    new CookieRequestCultureProvider
                    {
                        CookieName = "_cultureLocalizationGenially"
                    },
                    new QueryStringRequestCultureProvider(),
                    new AcceptLanguageHeaderRequestCultureProvider
                    {

                    }

                }
            };

            app.UseRequestLocalization(requestLocalizationOptions, defaultRequestCulture: new RequestCulture("en-US"));

Do you need any more? Thanks!

I will add some screen captures showing the differences...

I have added two screen captures, the same URL (with no caching). The slow has the cookie set, the  fast, no. Tested with Fiddler with Chrome, Safari, Edge and Firefox. Same result. The difference is too big.

@shanselman @hishamco any ideas or help?
Thanks.

sorry @chemotaxis I was sick last few days, my question again why you need to define the culture providers while they are already there?!!
If you need to change the cookie name, please use:
CookieRequestCultureProvider.DefaultCookieName = "_cultureLocalization";

modify what I said before and have a try

Hi @hishamco, I solved the problem... It is a problem with Resources files, if you don't have a default resources named like My.Resources.en-US.resx where en-US is your default request culture the system works, but it is very slow.
I had this:
My.Resources.resx
My.Resources.fr-FR.resx
My.Resources.it-IT.resx
My.Resources.pt-PT.resx
My.Resources.pt-BR.resx
My.Resources.de-DE.resx
Now I have this:
My.Resources.resx
My.Resources.en-US.resx
My.Resources.fr-FR.resx
My.Resources.it-IT.resx
My.Resources.pt-PT.resx
My.Resources.pt-BR.resx
My.Resources.de-DE.resx
Is this a bug?

I see, nothing but the sample that's provided by this repo is also missing the resource for the default culture. I'm not sure if this related to issue #162
/cc @Eilon  @ryanbrandenburg

@ryanbrandenburg let's see if we can repro this and perhaps run in a profiler to see what's taking so long.

@chemitaxis I'd like to inform you that the default resource issue is mentioned in the issue #257 too, hope the @ryanbrandenburg fix will solve the issue

@chemitaxis how slow were you seeing it? While I haven't actually profiled this, when replicating it in samples\LocalizationSample I didn't see a noticeable difference in speed between a missing and an existing Language. Does this still reproduce for you when using our latest packages? If it does, can you upload a complete minimum reproduction app to a GitHub repo?

Closing this one out as could not reproduce. If you still run into this please reopen with a reproduction repo.

Ok, close, but the problem persist, in Azure Web App. Thanks for all. ;)
Best.
2016-07-15 18:54 GMT+02:00 Ryan Brandenburg<EMAIL_ADDRESS>
Closing this one out as could not reproduce. If you still run into this
please reopen with a reproduction repo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aspnet/Localization/issues/252#issuecomment-233006682,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABWseenXI-jjzcIKV31gYRNoeY9lL69Hks5qV7tIgaJpZM4Isl5M
.

I'm not able to replicate your described behavior, even on an Azure Web App. Likely there's another factor which hasn't been mentioned here contributing to your problem. It would help me to find that factor and reproduce your problem if you could provide an entire project which exhibits the behavior (as a GitHub repo). If you can provide me with that I will investigate further.

Ok, I will try this weekend... Thank you so much for your time... And sorry if I was rude... It dit not intend. Best.

Hi @hishamco ,
I am trying to set the default culture cookie name and came across your suggestion of setting
CookieRequestCultureProvider.DefaultCookieName = "_cultureLocalization";
But how can this work with a static readonly?
In a controller-action, the following works
                Response.Cookies.Append(                  CookieRequestCultureProvider.DefaultCookieName,                  CookieRequestCultureProvider.MakeCookieValue(new RequestCulture(cultureName)),                  new CookieOptions { Expires = DateTimeOffset.UtcNow.AddYears(1) });
but otherwise setting the name does not work
                Response.Cookies.Append(                  "myCookie",                  CookieRequestCultureProvider.MakeCookieValue(new RequestCulture(cultureName)),                  new CookieOptions { Expires = DateTimeOffset.UtcNow.AddYears(1) });
Please advise if possible


Remove camel-case imageName runtime test

As a follow up of #2132 we decided to remove the camel-case test because it does not match any real world scenario. Beside this, a small return value by the mock is fixed.
This PR should decrease the unit test coverage a bit.

/ok-to-test

LGTM

/lgtm

@rhatdan needs /approve too

/approve

Thanks!


refactor: changed to commonjs file ext

What kind of change are you introducing

[ ] Breaking change
[ ] New framework implementation
[x] Misc. lib improvement

:tada: This PR is included in version 0.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


White / black list entities for intents

Currently you can instruct Rasa Core do ignore entities for certain intents:
intents:
  - greet: {use_entities: false}

We should enable a more detailed configuration for that, so that you can exactly specify which entities should be considered by an intent and which should be ignored.

@wochinge this should be covered by @mauricedoepkes PR right? so we can close this?


Update to 3.69.7 - piVCCU repeatedly forgets all devices

After upgrading to version 3.69.7, piVCCU  "forgets" all devices after a couple of hours or days. After a system reboot, the devices are back. Sadly, I do not remember what version was installed before. I also installed some other software together with the update, so I am not 100% sure what may be the reason of this behavior.
I run piVCCU together with openHAB ond a Raspberry Pi 3.
Find the relevant content of /var/log/messages attached. I suppose the 01:44:30 message is important, since I rebooted the pi after this.
Is there another place to find logs on raspbian OS? /var/log/hmserver.log and /var/hm_mode do not exist.
messages.txt

Hi folks,
same here. UI proclames: "HmIP-RF Eine Komponente der Homematic Zentrale reagiert nicht mehr."
I see the following warning on CCU side: "XmlRpc transport failed (first try), retrying..."
followed by the error "XmlRpcClient error calling event({[methodName:"event",params:{"BidCos-RF_java","CENTRAL","PONG","RPCAL02(rpcal02):hm-rpc.0:MYID"}]}) on http://<IP_ADDRESS>:39292/bidcos:--" and "XmlRpc transport error".
iobroker tries to connect and gets this error: "\Tools\Server\iobroker\rpcal02\nodejs\node.exe: 2023-08-21 08:29:47.078 - error: hm-rpc.1 (13292) Ping error [RPCAL02(rpcal02):hm-rpc.1:MYID]: Unknown XML-RPC tag 'TITLE'".
Please tell me if I can assist with logs or other stuff.
Best, Alex

Resolved for me since I upgraded to 3.71.12-85.


Review Torch Choice tutorial

Link to tutorial .
I will review and comment all the pages accessible in the link above
@TianyuDu and @kanodiaayush

Tutorial for Data Management

"Note: since this package was initially proposed for modeling consumer choices, attribute names of ChoiceDataset are borrowed from the consumer choice literature." To the extent it is possible, I would recommend keeping all the terms as generic as possible. Eg.

Consumer <-> user
Purchase record <-> Record
Purchase <-> choice

General remark 1: this page has "2" sections - one gives a high-level overview and implements examples. Personally, when searching for functions online I am used to first seeing the content of the function and then explanations and then examples. So I would recommend doing the same here. You can start with step 2 of your toy example - see screenshot below. I prefer this because I can read the variable names and guess what they are by they name. Then I would scroll down to get more information - eg. dimensions and type of "user_index", default values of "price_obs", etc.

Related to the above comment, I couldn't follow the "Advanced Usage: Additional Observables" paragraph without knowing what variables are in the package. For me, this paragraph would be clearer later in the page
About the "What you can do with the ChoiceDataset?" section. Do you have recommendations for doing descriptive stats? Should the researcher do them before passing them through ChoiceDataset? In particular given that everything is transformed into integers

Thank you for this helpful and constructive feedback, I truly appreciate these ideas! I am looking through all of them and identifying appropriate ways to address each of them.

I updated your comment by replacing bullet points with check boxes, this makes tracking progress easier.

I have updated the abuot page (now called introduction) here and the data-mangement page here.

BEMB Tutorial - link
There is a lot of information on this page and I'm not sure what is the best way to present it - it depends on what the reader is looking for. I'll make some suggestions but will mostly highlight what information I didn't find or understand. Hopefully this will give you ideas and we can also discuss this at some point.

I'm not sure we need an example in the introductory paragraph. Also, the one provided is both more and less general than what the package does: the cdf F is more general than Gumbel distribution but theta*alpha is less general than the utility functions the package can accomodate
Utility formula: you could be more specific about what utility representation the package allows for. I think (maybe I'm wrong) that

you only have logistic models (the noise epsilon is constrained to follow a Gumbel distribution)
utility function is additively separable in the observables and allows for interactions between latents. It should be stressed that this actually very a general form because (i) one can always build sophisticated observables, for instance by taking the log or a polynomial transformation of original observables and (ii) because one can impose that the learnable coefficients depend on i, u, s or any combination of them.

Utility formula: we need to be more specific about the model and review the math notation (which is currently incorrect).

for details, see page 376 of Athey et al (2021)
the model assumes unit demand for each category, independent choices across categories, and error term distributed according to Guembel distribution (logit)
there needs to be a discussion on how the outside option is modelled. How does the model choose that the user biuys nothing from a given category? Can we change the value of the outside option in each category or is normalized to 0 for each category?
Regarding notation: (i) need to index the variables by {uis} and (ii) decompose the utility into a deterministic part and the error term:
$\mathcal{U}{uis} = U_{uis} + \varepsilon_{uis}$.
Then $P(i|u,s)$ is a function of $U_{uis}$ instead of $\mathcal{U}_{uis}$
I suggest we write the utility function that the package can accommodate in its most general form (ie. sum all the terms that can be included) and then discuss each term one by one

Subsection "Specifying the Dimensions of Coefficients with the coef_dim_dict dictionary".

I didn't understand what point 4. refers to. I am not sure it was specified in the "Utility formula" subsection that there can be matrix factorization coefficients for observables

There is a section "Specifying Variance of Coefficient Prior Distributions with prior_variance" but I think there is no section about setting the mean of the coeffs.
Regarding obs2prior:

there should be a link to the dedicated tutorial in this subsection
it is not clear whether with obs2prior we impose that the variance is the identity matrix or if we can change that.
it is not clear whether we can impose some form to H or not

"If category_to_item is not provided (or None is provided), the probability of purchasing item i by user u in session s is ..."

maybe we could slightly rephrase into saying that by default there is only one category which is all the items. But the package can impose subcategories. In any case, the model is unit demand per category: the user buys at most one item per category

H_zero_mask Option in obs2prior
Link to notebook (use branch "mask-H-obs2prior")
This tutorial explains how to impose some structure on hierarchical priors. Specifically, it shows how to impose that some entries of the H and W matrix are fixed to 0 and shouldn't be learned. Here are some comments

I think this is a great extension for the obs2prior tutorial. I will use it in the Learning Tools project and, when this is done, we could add a link to the Learning Tools notebook to show a practical example
We could give some context regarding when imposing structure on hierarchical priors is relevant.

if I understood correctly, this is mostly relevant when the researcher fears there is not enough data for a flexible hierarchical prior and the researcher has some intuition regarding which dim - to be confirmed

We could also give some hints regarding how to choose the structure of the hierarchy
Related to the two points above, we could adapt the numerical example to highlight the benefits from imposing a structure

Tutorial for BEMB with Simulated Data and the obs2prior Option
In this post I review the notebook on obs2prior. Note that I had reviewed the H_zero_mask option which is an extension to this notebook in the post above. Here are some comments

[ ] we could add some context regarding when using observables helps and when it doesn't
[ ] Relatedly, it would be useful to add a summary of what the tutorial will do before we start the simulations. What are we trying to predict, what do we know about the underlying preferences, what variables do we observe, why in this context we expect using obs2prior will help, etc.
[ ] "The observable of a particular user is a one-hot vector with width num_items and one on the position of item this user loves (as mentioned previously)."

I thought that this was precisely what we don't observe and try to recover

[ ] For internal purposes: often, the user observables relate to demographics (age, gender, income, etc.). Why didn't we choose to simulate this type of situation?
[ ] "Fitting the Model"

it would be useful to talk more about what the package does and what input is necessary. For instance, do we need to set a prior for H and W, does the package sets them by default, does the package tries different ones and selects the best one?

I have remove the "about page" and created an more integrated "introduction page" here
I have also added an example of using LabelEncoder to encode raw item names to integers.

General remarks

I have found some of those invisible typos - eg. "Modelling" instead of "Modeling" - and as in any project there may be a bunch that are hard to find. I recommend installing Grammarly on your google chrome, and open the jupyter notebook on a website. Grammarly will spot all these typos and the misplaced commas, and sentences to be rephrased. A screenshot below of what I see when I write these lines

The first 4 pages - Home, About, Get started, and Tutorial for Data Management - often repeat what was said in the other pages. I would recommend specializing the pages and providing all the relevant information about the topic covered - ie. never let the reader think he found the page covering the topic he/she was looking for when it is not the appropriate page. Alternatively, provide links to the main page covering the topic that is mentioned in another page

[x]  eg. installation is mentioned both in Home and Get started but the latter provides more options for installation.
[ ]  eg. the About page introduces some models and notation but does not explain how they relate to the package, which is covered in the Tutorial for Data management page.

I will be brief on the first 3 pages because they will likely change a lot and I'm not an expert to introduce a package. I will give thoughts that crossed my mind

Home

[x]  I'm not sure we need to provide an example using the Canada example on the home page.
[x]  "1. The package includes a data management tool based on PyTorch's dataset called ChoiceDataset." It seems there is a typo or a word in missing in "PyTorch's dataset"
[x]  "4. For those without much experience in model PyTorch development ..." I think I am one of those and I felt there was many words in this paragraph that I didn't know/understand (eg. "optimizers", "training loops", "PyTorch lightning wrapper"). I don't know if you have added a feature to your package for people without experience or coded some technical parts using existing packages. Also, would you say that those with experience will use this feature as well or not?

About

[ ]  I think you can be more formal about the model + what is the objective function that is optimized + name the optimization technique.
[ ]  I am not sure that the Components of the Consumer Choice Modelling Problem are necessary in this page. They will be described in details in the Tutorial for Data Management page.
[ ]  In case you keep it, I feel that many terms and descriptions relate to consumers in a supermarket. You could rephrase everything with more generic terms and give examples using supermarket or learning apps. Examples below
[ ]  Consumer <-> user
[ ]  Purchase record <-> Record
[ ]  purchase <-> choice
[x]  "Note: since we will be using PyTorch to train our model, we represent their identities with integer values instead of the raw human-readable names of items". I like the "raw human-readable names of items" :) I think this note is important and shouldn't be lost in the middle of the page but could be in a dedicated section. In particular, you could add tips on how to work with integer values instead of strings. Perhaps in the tutorial there is code for building a dictionary? How can one do raw human-readable descriptive stats on their data when the key variables are integer values?

I have remove the "about page" and created an more integrated "introduction page" here
I have also added an example of using LabelEncoder to encode raw item names to integers.

Tutorial for Data Management

[X]  "Note: since this package was initially proposed for modeling consumer choices, attribute names of ChoiceDataset are borrowed from the consumer choice literature." To the extent it is possible, I would recommend keeping all the terms as generic as possible. Eg.

Consumer <-> user
Purchase record <-> Record
Purchase <-> choice
I don't know about "price" - it's a generic term but how is it treated differently than any other item characteristics? Is it actually necessary in the package?

The price variable does require some future elaboration: since price is a variable depending on both item and session, so by price variable I mean observables that depend on both item and session just like the price.  I agree this rises a lot of confusion. I have added some explanations in the new introduction page here. I am also looking for ideas on an alternative word for this kind of variables; probably we can just call it itemsession-obervables? Or we can call them (item, session)-observables? @kanodiaayush @charlespebereau

[ ]  Coming back to the "Components of the Consumer Choice Modelling Problem" section after reading the whole page: i terms of presentation, I think you shouldn't write about context the same way you write about parameters of the package. For instance, I was confused I couldn't find any "Purchase record" in ChoiceDataset().

[ ]  General remark 1: this page has "2" sections - one gives a high-level overview and implements examples. Personally, when searching for functions online I am used to first seeing the content of the function and then explanations and then examples. So I would recommend doing the same here. You can start with step 2 of your toy example - see screenshot below. I prefer this because I can read the variable names and guess what they are by they name. Then I would scroll down to get more information - eg. dimensions and type of "user_index", default values of "price_obs", etc.

[ ]Related to the above comment, I couldn't follow the "Advanced Usage: Additional Observables" paragraph without knowing what variables are in the package. For me, this paragraph would be clearer later in the page

[ ]About the "What you can do with the ChoiceDataset?" section. Do you have recommendations for doing descriptive stats? Should the researcher do them before passing them through ChoiceDataset? In particular given that everything is transformed into integers

Currently, we want users of our package to conduct their own descriptive analysis of their dataset using their own favorite analysis tool. I agree that our ChoiceDataset could have a succinct helper function providing simple summary statistics of the dataset. I have initiated a new issue here for this.

[ ]  "Overall, there are four types of additive component, except the error term, in the utility

notation: specify the dimensions of X_i^item
could we add a component with user observables X_u^user?

For improvements on the BEMB documentation website, I have moved these comments to the bemb repository (please see links above).

All feedback by Charles about the torch-choice package has been addressed in the latest documentation website. All comments for the bemb package documentation has been copied to issues in the BEMB repository.
Thanks for this really constructive feedback!
@charlespebereau @kanodiaayush


Allow similar syntax for scheduling labels that "Selector Labels" allow

Hey there,
i run into a problem. I was expecting that this http://docs.rancher.com/rancher/labels/ (Selector Labels) will work for service scheduling too. But it won't. Like this
mongo-replSet-rs0:
  container_name: mongo-replSet-rs0
  image: mongo:3.0
  command: mongod --config /mongod.yml
  stdin_open: true
  tty: true
  restart: always
  ports:
    - "27017:27017"
  volumes:
    - /mongo:/data:rw
    - /data/mongo/mongod.yml:/mongod.yml:ro
  labels:
    io.rancher.scheduler.affinity:host_label: mongo in (primary, secondary, arbiter, hidden)

Another solution for me would be to be able to only specify key without value.
Is this possible somehow?
Edit: After carefully reading the docs again I saw Key Value Pair of Host Label specified for affinity. My fault. But is there a way?

After some unsuccessful research i guess not.
I make this a feature request.

Correct, the syntax for selector labels is only applicable for selector labels.

With the release of Rancher 2.0, development on v1.6 is only limited to critical bug fixes and security patches.


Requesting XCode 12, iOS 14 Support

With the release of new iOS 14 (XCode 12 Beta) the current stable version 1.9.0 doesn't compile with carthage command.
v1.9.0 Commit: 7f36441e3372665b1b414f8ac93b5905cc42a405
Carthage Command: carthage build --no-skip-current --platform ios
  var readBuffer = UnsafeMutablePointer<UInt8>.allocate(capacity: 1)
  ~~~ ^
  let

** ARCHIVE FAILED **

The following build commands failed:
	CompileSwift normal armv7
	CompileSwiftSources normal armv7 com.apple.xcode.tools.swift.compiler
	CompileSwift normal arm64
	CompileSwiftSources normal arm64 com.apple.xcode.tools.swift.compiler
(4 failures)

However the master branch compiles perfectly with above carthage command.
Wondering what the timelines are for next release?

I'll look at cutting a release later this week/early next week.

Thanks @thomasvl :) Will keep an eye out for new release.

1.10.0 released


middleware/file: export dnssec metrics

signature expiration would be nice.

No!
this is stuff a 3rd party scraper should check and export.


How to handle multiple scrapy processes using TorIpChanger?

Hi,
Thanks for this excellent tool which works perfectly fine on my ubuntu server.
I have a question, do I need to use multitor (https://github.com/trimstray/multitor) to run multiple instances?
Now I am currently running about 20 processes for crawling different sites, in each process there is method to call get_new_ip() after 10 requests reached, which I think this may not be good or best practice.
If I only get one machine, and running 20 processes, should I remove calling get_new_ip in each process and use TorIpChanger seperately, like to make a crontab to change the exit ip every minute?
Really appreciate this lib and have a nice day!
BR

do I need to use multitor (https://github.com/trimstray/multitor) to run multiple instances?

Yes. Another alternative is https://github.com/mattes/rotating-proxy. I was also thinking about writing a docker-compose and TorIpChanger based rotating Tor proxy but didn't get to it ...

Now I am currently running about 20 processes for crawling different sites, in each process there is method to call get_new_ip() after 10 requests reached, which I think this may not be good or best practice.

Well, personally I think changing IP after 10 requests is too often.
Do I get it right you scrape those 20 sites simultaneously? If that's the case, I would somehow sync the requests, meaning I would change IP only after requests to all 20 sites were handled. This way each of those sites will be scraped from a different IP and at the same time you wont hit the fast responding sites too much as you will have to wait for the slowest one. You shouldn't hammer the target site too hard in general (sleeps between requests, etc)
Your suggestion of using a cron job may result in changing IP mid request and I don't know if you would get the response back [I don't know if Tor waits till all requests are finished before changing circuit].

Yes, simultaneously scraping 20 sites on one server.  Thanks for your suggestions and I'll experiment on that ^^
Thank you.


Latest Insiders uses nodejs 16 and may have broken extension

I think they finally moved to nodejs 16 and the streams interface I was using changed:

Upstream: https://github.com/microsoft/vscode/issues/144962

Fixed in bd55931a55cd1154f3c917d98bb20884bf6c1db3 using the generic pipeline argument rather than arrays.


Cannot close a folder

Type: Bug
I have a folder open that has five or six files and 3 subfolders. I want to close the folder to start a new project. I only have this folder open in one window. There's no option to close a folder in my File menu. Just Open File, Open Folder, Open Workspace from File ..., and Open Recent. The only Close option is Close Editor. Google told me to Ctrl-K-F and that has worked for me in the past but now it either does nothing or messes with a css file (when it was still open). I tried closing the whole program and reopening it but the folder is still open. I checked Updates and there are none.


testQuick (or incremental testing) is not supported

I haven’t actually tried, so I might be totally wrong, but a quick googling of “minitest testQuick” gave no result.

testQuick I think works for anything that has test-interface integration. Here I just change one source, and it ran one test suite only:
[info] Done compiling.
example.sourcecodetest.Apply
- Name
- FullName
- Enclosing
- Pkg
- SourceFilePath
- Line
- lazy Name
- lazy FullName
- lazy SourceFilePath
- lazy Line
- lazy Enclosing
[info] Passed: Total 11, Failed 0, Errors 0, Passed 11
[success] Total time: 1 s, completed Aug 7, 2019 2:19:17 PM
3. Waiting for source changes in project minitestRoot... (press enter to interrupt)


Fix window maximize error message appearing when starting the web editor

master version of https://github.com/godotengine/godot/pull/62522.
Follow-up to https://github.com/godotengine/godot/pull/61460.

I've noticed that other platforms (e.g. Android) do not print the warning (just completely ignore the maximization). I'm wondering if we should try to stay consistent and simply drop the WARN_PRINTs in the HTML5 platform in this cases.

It turns out this PR is unnecessary in master, as that warning message only exists in 3.x.


Tag you release workflow with action to a working commit

What this PR does / Why we need it
Fixes your release workflow https://github.com/actions-ecosystem/action-get-latest-tag/runs/6008122338?check_suite_focus=true#step:8:1

no longer needed after https://github.com/actions-ecosystem/action-get-latest-tag/pull/19#issuecomment-1098104634

no longer needed after https://github.com/actions-ecosystem/action-get-latest-tag/pull/19#issuecomment-1098104634


Synthesis failed for nodejs-projectify

Hello! Autosynth couldn't regenerate nodejs-projectify. :broken_heart:
Please investigate and fix this issue within 5 business days.  While it remains broken,
this library cannot be updated with changes to the nodejs-projectify API, and the library grows
stale.
See https://github.com/googleapis/synthtool/blob/master/autosynth/TroubleShooting.md
for trouble shooting tips.
Here's the output from running synth.py:
s/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/googleapis/nodejs-projectify
2021-04-17 05:37:02,792 autosynth [ERROR] > Synthesis failed
2021-04-17 05:37:02,797 autosynth [INFO] > Changed files:
2021-04-17 05:37:02,797 autosynth [INFO] >
2021-04-17 05:37:02,797 autosynth [DEBUG] > Running: git log b33b0e2056a85fc2264b294f2cf47dcd45e95186 -1 --no-decorate --pretty=%s
2021-04-17 05:37:02,801 autosynth [DEBUG] > Running: git log b33b0e2056a85fc2264b294f2cf47dcd45e95186 -1 --no-decorate --pretty=%b%n%nSource-Author: %an <%ae>%nSource-Date: %ad
2021-04-17 05:37:02,804 autosynth [DEBUG] > Running: git add -A
2021-04-17 05:37:02,807 autosynth [DEBUG] > Running: git status --porcelain
2021-04-17 05:37:02,811 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 29abfd0 refactor(nodejs): move build cop to flakybot (#237)
2021-04-17 05:37:02,814 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2021-04-17 05:37:02,818 autosynth [DEBUG] > Running: git diff --binary HEAD..autosynth-91 -- . :(exclude)synth.metadata
2021-04-17 05:37:02,821 autosynth [DEBUG] > Running: git diff HEAD..autosynth-91 -- synth.metadata
2021-04-17 05:37:02,824 autosynth [DEBUG] > Running: git diff --binary autosynth-91..autosynth-93 -- . :(exclude)synth.metadata
2021-04-17 05:37:02,827 autosynth [DEBUG] > Running: git checkout 29abfd0c00e8278d4417282e95dff59238ee7f84
Note: checking out '29abfd0c00e8278d4417282e95dff59238ee7f84'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 29abfd0 refactor(nodejs): move build cop to flakybot (#237)
2021-04-17 05:37:02,831 autosynth [DEBUG] > Running: git checkout c6706ee5d693e9ae5967614170732646590d8374
Previous HEAD position was b33b0e2 build(node): remove unused tokens (#1046)
HEAD is now at c6706ee fix(node): default-branch should use $default_branch (#1052)
2021-04-17 05:37:02,838 autosynth [DEBUG] > Running: git branch -f autosynth-92
2021-04-17 05:37:02,841 autosynth [DEBUG] > Running: git checkout autosynth-92
Switched to branch 'autosynth-92'
2021-04-17 05:37:02,845 autosynth [INFO] > Running synthtool
2021-04-17 05:37:02,845 autosynth [INFO] > ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2021-04-17 05:37:02,845 autosynth [DEBUG] > log_file_path: /tmpfs/src/logs/nodejs-projectify/92/sponge_log.log
2021-04-17 05:37:02,847 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --
2021-04-17 05:37:03,066 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-projectify/synth.py.
On branch autosynth-92
nothing to commit, working tree clean
2021-04-17 05:37:03,196 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-projectify/synth.py", line 10, in <module>
    templates = common_templates.node_library()
  File "/tmpfs/src/github/synthtool/synthtool/gcp/common.py", line 301, in node_library
    return self._generic_library("node_library", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/common.py", line 65, in _generic_library
    kwargs["metadata"]["repository"]
  File "/tmpfs/src/github/synthtool/synthtool/gcp/common.py", line 374, in _get_default_branch_name
    github_req.raise_for_status()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/googleapis/nodejs-projectify
2021-04-17 05:37:03,290 autosynth [ERROR] > Synthesis failed
2021-04-17 05:37:03,294 autosynth [INFO] > Changed files:
2021-04-17 05:37:03,294 autosynth [INFO] >
2021-04-17 05:37:03,295 autosynth [DEBUG] > Running: git log c6706ee5d693e9ae5967614170732646590d8374 -1 --no-decorate --pretty=%s
2021-04-17 05:37:03,298 autosynth [DEBUG] > Running: git log c6706ee5d693e9ae5967614170732646590d8374 -1 --no-decorate --pretty=%b%n%nSource-Author: %an <%ae>%nSource-Date: %ad
2021-04-17 05:37:03,301 autosynth [DEBUG] > Running: git add -A
2021-04-17 05:37:03,304 autosynth [DEBUG] > Running: git status --porcelain
2021-04-17 05:37:03,308 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 29abfd0 refactor(nodejs): move build cop to flakybot (#237)
2021-04-17 05:37:03,311 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2021-04-17 05:37:03,315 autosynth [DEBUG] > Running: git diff --binary HEAD..autosynth-92 -- . :(exclude)synth.metadata
2021-04-17 05:37:03,318 autosynth [DEBUG] > Running: git diff HEAD..autosynth-92 -- synth.metadata
2021-04-17 05:37:03,321 autosynth [DEBUG] > Running: git diff --binary HEAD..autosynth-93 -- . :(exclude)synth.metadata
2021-04-17 05:37:03,325 autosynth [DEBUG] > Running: git diff --binary HEAD autosynth-93
2021-04-17 05:37:03,328 autosynth [DEBUG] > Running: git apply /tmpfs/tmp/tmpwezp3tw_/autosynth-93.patch
2021-04-17 05:37:03,331 autosynth [DEBUG] > Running: git add -A
2021-04-17 05:37:03,334 autosynth [DEBUG] > Running: git status --porcelain
2021-04-17 05:37:03,338 autosynth [DEBUG] > Running: git commit -m chore: Fix python owlbot post processor image cloudbuild.yaml

I've confirmed that this works locally by running

gcloud builds submit --config=docker/owlbot/python/cloudbuild.yaml --substitutions=SHORT_SHA=$(git rev-parse --short HEAD)

Then I ran `python -v` in the docker container using

docker run --rm -it --entrypoint /bin/bash

Source-Author: Anthonios Partheniou<EMAIL_ADDRESS>Source-Date: Fri Apr 16 14:36:04 2021 -0400
Source-Repo: googleapis/synthtool
Source-Sha: ff6b759fcb58f216f62b48d5be0bb85fafbc3bb2
Source-Link: https://github.com/googleapis/synthtool/commit/ff6b759fcb58f216f62b48d5be0bb85fafbc3bb2
[autosynth a397ccd] chore: Fix python owlbot post processor image cloudbuild.yaml
 3 files changed, 3 insertions(+), 43 deletions(-)
2021-04-17 05:37:03,345 autosynth [DEBUG] > Running: git log -1 --no-decorate --format=%B
2021-04-17 05:37:03,348 autosynth [DEBUG] > Running: git push --force origin autosynth
To https://github.com/googleapis/nodejs-projectify.git
 + ef798e2...a397ccd autosynth -> autosynth (forced update)
2021-04-17 05:37:05,420 autosynth [DEBUG] > Running: git log -1 --pretty=%b
2021-04-17 05:37:07,752 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing build/
Removing node_modules/
Removing package-lock.json
Removing samples/node_modules/
Removing samples/package-lock.json
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 356, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 191, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 336, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 95, in synthesize_loop
    synthesize_inner_loop(fork, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 139, in synthesize_inner_loop
    synthesizer, len(toolbox.versions) - 1
  File "/tmpfs/src/github/synthtool/autosynth/synth_toolbox.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Autosynth passed, closing! :green_heart:


JDBC4ResultSet.getClob() should return null if the data is null

The JDBC4ResultSet::getClob methods never return null, even when the ResultSet's string data is null:
https://github.com/xerial/sqlite-jdbc/blob/3d04d7df0c89240add2c92189adb30b6cb7e6ae0/src/main/java/org/sqlite/jdbc4/JDBC4ResultSet.java#L378-L384
That's quite contrary to the expectation, or the behaviour of any other JDBC driver I've seen so far. The expectation is for the method to return a null value. The Clob API doesn't have any good ways to check for content null-ness. The only thing that could be done is to check one of the stream methods (e.g. Clob::getCharacterStream) if that returns null.

The expectation is for the method to return a null value

Hi Lukas, would you be able to share a bit more on that expectation? The ResultSet interface doesn't mention anything.
I know you have worked with pretty much all the drivers out there, but it might be good to share a bit more if you can.

Indeed, neither the ResultSet Javadoc, nor the JDBC spec are explicit about this, regrettably. Perhaps take the CallableStatement::getClob Javadoc as an inspiration? I don't see a reason why the two behaviours should be different in those cases. It says:

the parameter value as a Clob object in the Java programming language. If the value was SQL NULL, thevalue null is returned.

Think about it this way: All non-primitive type returning ResultSet.getXYZ() methods return either a null value meaning SQL NULL, or some non-null value meaning SQL NOT NULL. Except this particular JDBC4ResultSet.getClob() method, which never returns null in sqlite-jdbc's case. So, how can we detect whether the Clob value had been null?
We could call ResultSet::wasNull, of course, but that seems weird, because the simplest way for a ResultSet to indicate that a returned reference type was null is simply to return null. We could call some derived method, like Clob::getCharacterStream, but that seems kind of clever. If the java.sql.Clob API (and the other resource wrappers, including Blob, Array, SQLXML) were meant to contain SQL NULL values, then there would be a Clob::isNull method, in my opinion.


[kafka] Implement KeyValue support for KafkaBytesSource

Motivation
Add support for different key and value Deserializers for the Kafka Source.
With this change the Kafka Connector supports non-String keys and it also apply the correct Schema to the Pulsar Topic.
For primitive datatypes we are not decoding the Kafka key pair into Java Objects, we are simply passing a reference to the internal ByteBuffer (that is a wrapper for a byte[]).
The Schema type is decided using the keyDeserializationClass and valueDeserializationClass parameters that you pass to the Kafka Source configuration.
This is the mapping;

ByteArrayDeserializer,ByteBufferDeserializer,BytesDeserializer:  Schema.BYTEBUFFER
StringDeserializer.class: Schema.STRING
DoubleDeserializer:  Schema.DOUBLE
FloatDeserializer: Schema.FLOAT
IntegerDeserializer: Schema.INT32
LongDeserializer: Schema.INT64
ShortDeserializer: Schema.INT16;
KafkaAvroDeserializer: Schema.AVRO (schema is downloaded from the SchemaRegistry)

When the key deserializer is StringDeserializer we use the decoded key as Pulsar key.
When the key is not StringDeserializer then we use the Pulsar KeyValue data type, with SEPARATED key encoding .
This way on the topic we have a Schema for the key and a Schema for the value.
The key is encoded into the Pulsar key (SEPARATED) and so it is used for routing and for compaction.
Limits of this patch:

we are not supporting byte[] keys (this is a limitation of Pulsar IO Record, that is to be addressed separately)
we are only supporting SEPARATE KeyValue encoding (adding support for INLINE is simple, but can be done in a separate change if some user requests such support)
there is no support for JSON payloads (this is to be implemented in a follow up work)

Modifications

refactor AbstractKafkaSource  in order to

Verifying this change
The patch introduce unit tests that cover the new code.
Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? it should be documented with dedicated docs

Can someone help me reviewing this patch ?

@congbobo184 @gaoran10 Could you please also help review this PR?

Thank you @codelipenghui  and @congbobo184 I have merged this patch.
I will follow up with other enhancements to the KafkaSource


Thank you from the Slovenian Computer History Museum

Our IBM PC died (understandably and it has well earned its rest) but we wanted to continue exhibiting it, so we used a Raspberry Pi to drive its IBM 5153 CGA monitor using this project: https://github.com/trylle/ibm515x/issues/3
It's a great hack to drive a CGA monitor but the resolution it uses is 1920x200 (probably for compatibility with RPi GPU timings etc...) - normal DOSbox(-X) does not scale well so we used VirtualXT browser version and made it stretch to fill screen. It works great!
And this is the software we're showcasing - a Slovenian program developed in BASIC to measure (and playback data) of the reactivity in the core of our country's one and only nuclear reactor! ☢

Thank you for sharing!
That is a very cool setup I must say. :smile:
It is always very encouraging when people find use for your software. Happy that it could be useful. Let me know if there is anything you need or would like to see added in the VirtualXT project. I see you had to do some adjustments to the script.js file in order to get it to scale. I can see if it is possible to fix or expose some parameters so your users do not need to edit anything.
One thing that is coming in version 0.9 is ISA passthrough support via the CH367 or the ARS Tech USB2ISA adapter. So using the ARS Tech adapter you should be able to also connect real ISA cards to the RespberryPI and have them show up in VirtualXT just like they would in a real machine.

Thanks for quick reply :D

I see you had to do some adjustments to the script.js file in order to get it to scale. I can see if it is possible to fix or expose some parameters so your users do not need to edit anything.

I think this is a very niche use which not many would need implemented, I just shared the information for anyone implementing a similar project  - might be just a few people since https://github.com/trylle/ibm515x has no activity :)
Great about the ISA passthrough, your emulator becoming quite powerful!


Update existing MD from new module syntax

This is to take new syntactical updates (e.g. a new parameter) to already-documented cmdlets in the Markdown file.

dup for #38


Checkbox Accessibility Tab updates and DAP violations

This issue is related to #491
The following updates are needed to the Checkbox Accessibility Tab:
1. Remove the first paragraph:
"The Checkbox React Carbon component has been tested against the latest W3C Web Content Accessibility Guidelines (WCAG) 2.1 Level A and AA success criteria and violations have been identified as high priority issues. This document will be updated when these accessibility issues are resolved."
Note: This paragraph is part of the Non-compliant template and should not be added to the completed documents. The non-compliant template is used when the document can not be completed due to open accessibility issues that block completion.
2. Typically a different font is used for navigation keys. Is there a Carbon font that can be used? If there is, the following navigation keys should be updated in the second paragraph under how it works:
Tab
Space
3. Change the font to represent code for aria mark-up ** in the second paragraph under how it works:
aria-checked="true"
aria-checked="false"
aria-checked="mixed"
4. Under the Heading "Accessibility Testing" add the following paragraph:
Accessibility issues are tracked in the Carbon Component GitHub repository.
5. Update Automated Test Environment > Results because new DAP violations were reported.  Change to: "DAP Test - Violations"
6. The following DAP violations are also found. Let me know if they need to be reported on a separate issue (see screenshot below):

@snidersd could you report the DAP violations in a separate issue under this repo. Let's keep this issue to track docs.

@aledavila @dakahn - Can you confirm with DA. We discussed the doc issues this morning and he advised me to keep the issues together. Also note, there are DAP violations specific to the documentation. Thx


Directories not found in two pushd commands in test-all.sh (release v2.9.0)

Line 16 has pushd stan and line 24 has pushd stan/lib/stan_math_2.7.0, but the directory is actually named stan_2.9.0, and not stan.
The shell script seems to be running fine after changing the two lines to pushd stan_2.9.0 and pushd stan_2.9.0/lib/stan_math_2.9.0.
#130

Thanks. I'll get that fixed now.


⚠ Bofrost.es has degraded performance

In a943a79, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1958 ms

Resolved: Bofrost.es performance has improved in c920bc7 after 1 hour.


Blackmagic Hyperdeck Loop Feature

Would it be possible to have a loop in and loop out feature added for the BMD hyperdeck studio mini. Looking in the hyperdeck manual it appears that mark in and mark out can be made but only with timecode and I have has a go but not able to find a way of getting 2 timecode points to store in a command at once (1 for the mark in and 1 for the mark out).
I have been using a hyperdeck with the latest firmware and also using Companion 2.1.2
Thanks and loving the work everyone is putting into this!

Yes, this should be possible. A new action could allow you to set in and out points (as timecode), with another action to clear the playrange.

Link to my comment in what could be a related issue, or at least solved with a common solution:
https://github.com/bitfocus/companion-module-bmd-hyperdeck/issues/53#issuecomment-995203428

This is also related to #12


Extending DeleteTest test cases

added few test cases into the DeleteTest class

+1 if all checks pass.


Using GPU for simpleHTR

Python version: 3.5.4
Tensorflow version: 1.9.0
Operating system: Windows 10
Is a GPU implementation provided? I have a GPU so I wanted to train the dataset on that.
Thanks.

Both cpu and gpu work with the same model. It just depends on your Tensorflow installation.


Improper license

Following the transfer of ownership from Cityzen Data to OVH, the license has wrongly been changed from Apache 2.0 to a BSD like, this is in violation of the original license and is a change which must be reverted ASAP.

revert done on the #23

Perfect, thanks for the quick response.


[DRAFT] Feature branch for device and address reconciliation aimed at v1.16

This is not intended to be merged and is open only for running CI. We'll upstream commits from this branch in separate PRs. Periodic merges from main back to this.
Patchsets to upstream:

[ ] Generic reconciler (@joamaki, https://github.com/cilium/cilium/pull/30303)
[x] statedb improvements (@joamaki, https://github.com/cilium/cilium/pull/30246)
[ ] hive/job improvements (@dylandreimerink)
[ ] GetDevices()&friends removal (@bimmlerd)
[ ] Loader modularization (@dylandreimerink)
[ ] Bandwidth manager qdisc reconciler (@joamaki)
[ ] sysctl reconciler (@pippolo84)
[ ] Service NodePort reconciler (@joamaki, https://github.com/cilium/cilium/pull/30300)

Extras not part of the devs&addrs work that we'll upstream later:

[ ] BPF map operations & batch operations
[ ] Kubernetes statedb reflector
[ ] stream.Buffer

Cool things to check out:

pkg/statedb/reconciler/example (generic reconciler)
pkg/statedb/reconciler/benchmark (benchmark for reconciler + statedb)
pkg/statedb/reflector/k8s.go (k8s to statedb reflection)
pkg/bpf/ops/example (statedb to BPF map reconciliation)
pkg/datapath/linux/sysctl (sysctl reconciler)

Remaining work:

[x] Removal of option.Config.GetDevices(), AreDevicesRequired() (@bimmlerd)
[ ] Removal of option.Config.DirectRoutingDevice, DirectRoutingDeviceRequired (@bimmlerd)
[ ] Move IPv6MCastDevice logic to pkg/mcast (@bimmlerd)
[ ] Removal of DeviceManager (@bimmlerd)
[x] Removal of node.ipv[46]MasqAddrs (@joamaki)
[ ] Replace uses of NodeAddressing.{LocalAddresses,LoadBalancerNodeAddresses} with Table[NodeAddress] (@joamaki)
[ ] Removal of NodeAddressing (@joamaki)
[ ] Simple loader reconciliation loop (@dylandreimerink)
[ ] Simple iptables reconciliation loop (@pippolo84)
[ ] Reconcile address changes in pkg/service (@joamaki)
[ ] L2 announcements should watch Table[Device] (@dylandreimerink)
[ ] Remove daemon/cmd/device-reloader.go (@bimmlerd)
[ ] e2e/unit tests for validating reconfiguration on device/address changes
[ ] Replace LinkList/AddrList etc. calls with lookups to Table[*Device]? As follow-up?
[ ] Upstream all the things (everyone)

/ci-runtime

/test


[Xcode] React native run script build phase fails

I faced this issue in my project which I upgraded, but I created a new project using react-native init and I'm seeing it there as well.
../node_modules/react-native/packager/react-native-xcode.sh

Fails saying node is not found. I fixed it by changing it to the following.
source ~/.nvm/nvm.sh
../node_modules/react-native/packager/react-native-xcode.sh

+1 for this and thanks for the fix

+1

dup https://github.com/facebook/react-native/issues/3948

Please try react-native 0.14.2, which should include a fix for this.


Update Set-CsCallQueue.md

Adding new AuthorizedUsers parameter

@get-itips @dariomws @tseward @tiburd Please and thanks

Copy review - @CLYVR, kindly commit the suggested changes. Many thanks.

@CLYVR @JohanFreelancer9
Thanks for the edit pass, Johan - I committed slightly adjusted edits based on your suggestions.

@tiburd Please merge this PR. Thanks

@tiburd Is there anything more needed from me on this or can this be merged?  Thanks!

@CLYVR
Merging now. Sorry this fell through the cracks around the time I switched to a new device and Outlook format.....


Run Benchmarking utility in devnet environment

#3043 seeks to introduce the basic benchmarking infrastructure which has now given us benchmarks against a local fuel-core node in a Node.js environment (for browser support see https://github.com/FuelLabs/fuels-ts/issues/3136)
We should also run these benchmarks in devnet, also introducing blob/upload and upgrade transactions.

I think this could use a similar approach to how we do diffs of code coverage: by having two sets of results, one for the master branch and one for the current PR/commit—something like that. If we can upload/retrieve master bench results, we don't need to run things twice.