Using ngf-resize cause ngf-validate to fail. And resize created file name blob without file extension.

The codepen here demo the issues: http://codepen.io/hughred22/pen/NGaXXg?editors=101

upload image file that smaller than 1400 in width will not trigger the error message anymore. Get rid of ngf-resize, everything will work again.
upload image pass the validation, when it hit my server and save. The server file name will be number_blob without file extension like .jpg
See my codepen how I send the file. The photo property will be:
photo: "https://actorreels.stamplayapp.com/upload/cobject/pictures/1444892747388_blob"
Get rid of resize, it will be number_filename.jpg.

I exam the $file object, with resize, it called blob. The proto do not have get name function. Without resize, it called file and the proto has function get name... How can I change the file name into the name with correct extension?
Test image:

the picture you attached here has the size within the validation so I am able to select it and it will be set to the model. If I use smaller image it would fail validation and will not be set as model. If you have a file that is smaller and is passing the validation attach it here.

Blob has the 'name' property with the correct name of the file so it will be uploaded with the correct file name. For the multipart form-data request blob and file are the same thing.

The smaller image does not trigger the validation and show warning message using ngMessages. Check this again: http://codepen.io/hughred22/pen/NGaXXg?editors=101
I made 2 upload inputs now. The first one with resize. When you upload a smaller image, nothing happen and the line still blue and the createActorPageForm.pictures.$error do not trigger...
The second one without resizing. You see upload a smaller image will trigger the ngMessage - show warning and the line become red.

test image:

For 2) I am using stamplay as BAAS. They said they will put a timestamp and then the filename as the name. So they read whatever filename is the resize object blob. I can see blob has the name value. But what is the different between file and blob? File has get name in it prototype and blob do not. And Stamplay cannot get the correct filename for blob. Even I use Upload.rename.

Issues should be fixes in 9.0.11

I just tried 9.0.12. The issue did not get fixed. With my upload directive like this:
<input id="pictures" label="Upload Images" name="pictures" ngf-select="admin.upload($file)" ng-model="pictures" accept="image/*" ngf-max-size="5MB" ngf-validate="{width: {min: 1400, max:2000}, height: {min: 630, max: 900}}" ngf-resize="{width: 1600, height: 720, quality: 1.0}">

Now with the new update, validation is totally by passed by ngf-resize. Image with width like the second image I provide got resized and upload as well. Before, it will not get upload - but also will not trigger $error for ng-message. Now, it does not trigger ng-messages $error + it got upload to the server.
Second, I did not see the blob get name problem solved. Nothing is changed. The filename still did not get read by Stamplay. As you see in the screenshot, it still do not have the get name function in prototype like file: (see Object > data > photo: path -- still end up with _blob with no file extension)

Alright there was a bug introduced in 9.0.9 that made the validation pass silently.
Try it with 9.0.13, I also added name to the Blob prototype.

We are experiencing the same issue in 12.2.3
File name is blob. no extension.

Could you create a jsfiddle @r7lemieux ?


Module has no exported member 'promisifyRoutine'.

I believe the type definition is missing an export for promisifyRoutine:
Module '"node_modules/redux-saga-routines/src/index"' has no exported member 'promisifyRoutine'.

In next version type definitions will be removed from the repo.


Rodeofx fix cmx 3600 multiple markers per clip issue 593

This resolves conflicts in #597.

@elabrosseRodeofx This just fixes the merge conflicts from your PR.

Codecov Report

Merging #664 into master will not change coverage by %.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #664   +/-   ##
=======================================
  Coverage   81.69%   81.69%
=======================================
  Files          72       72
  Lines        2731     2731
=======================================
  Hits         2231     2231
  Misses        500      500

Flag
Coverage Δ

#py27
81.67% <ø> (ø)

#py36
81.67% <ø> (ø)

#py37
81.67% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0303169...d78c389. Read the comment docs.


[Store] No proper validation for incorrect password when logging into store

Description:
When giving wrong password it redirects to a blank page not showing any error messages.
Suggested Labels:

Suggested Assignees:

Affected Product Version:
APIM 3.0.0
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:

latest 3.0.0 pack it shows the proper error message.


materialization with reset_parameters

This implements materialization with reset_parameters (similar to how it is done in distributed) in the MaterializationTransform.

This needs to handle shared parameters (the other init methods probably, too).


Goto symbol: highlight symbol at current cursor location when opening

I posted earlier at https://github.com/Microsoft/vscode/issues/63306
I didn't update due to busy for a while, so the issue got closed by the bot.
When Quick Open is invoked first time for searching a symbol in active editor ( in my case, c/c++ editor),
Mostly tt seems better focus on the symbol where cursor is.

/duplicate of

/duplicate of https://github.com/microsoft/vscode/issues/22689


Add pre commit hook for cargo fmt

Let's use cargo fmt as a standard, so we avoid problems with formatting.
To avoid people forgetting, let's add a pre-comit hook.

Hey, since the path of commit hooks cannot be specified (before git 2.7) and it's inside .git, at the end we just need to use the same commit hook. I have created a gist for hosting it a while ago. https://gist.github.com/gonzamontiel/a9b3e60d47e3795a579fe27f4fa3877e#file-cargo-fmt-pre-commit-hook


feat(ModularForm): Graded Ring instance on spaces of modular forms

This adds the graded ring instance to spaces of modular forms.
Co-authored-by: Ruben Van de Velde<EMAIL_ADDRESS>
The porting of this was done with help by Ruben Van de Velde

Thanks!
bors d+

bors r+


Crontab-friendly script

So, basically this will let the developers run their commands from crontab easily.
This essentialy is a solution for https://github.com/akalongman/php-telegram-bot/issues/156

But I always agree with them, they always are an improvement I just didn't came up with!

Finally gotten around to test this... Nice work!
I see that it doesn't add anything to the DB, obviously, as it gets an update ID of 0.
Was just wondering if there might be a situation where one would want that. Not sure about implementation then to keep it nice and separate from the main updates.
Also, it would be great if you could make a short wiki entry explaining what this does.
Namely, that this cron can be used to have the bot send messages to itself.
Regarding the short command example you give at the top of the cron.php file, it would be great to put more meaningful ones, as sending /whoami or /echo vegetable doesn't really do anything.
I assume you use this to call maintenance related commands?
Apart from that I'm happy to merge this, will wait for your response 👌

(...)
Was just wondering if there might be a situation where one would want that. Not sure about implementation then to keep it nice and separate from the main updates.

I have no idea why would someone want to put 'fake' updates coming from crontab into DB, at first I've been using setCustomInput and sent fake update to run my scheduled commands, but it was literally flooding db with useless things (running 3 commands every minute),

Also, it would be great if you could make a short wiki entry explaining what this does.

Will do!

Regarding the short command example you give at the top of the cron.php file, it would be great to put more meaningful ones, as sending /whoami or /echo vegetable doesn't really do anything.
I assume you use this to call maintenance related commands?

Well, I didn't want to put non-existing commands there as an example...

Awesome wiki entry, very nice!


Issue 4 create basic eks cluster

Right now this is deploying nodes to the public subnets rather than the private ones. So still working on that.

Hitting a bit of a wall on the private/public subnet thing. The problem is that there are 2 pulumi apis for working with eks clusters, classic and something newer which seems to be related to the newer "aws native" stuff or the "crosswalk" stuff -- really hard to tell. The stuff you're noticing with the properties to set public and private subnet ids, is only available on the newer api. I noticed all these different APIs last week and was a bit distressed by it.
I see a few patterns:

All of the code sample project  are in "classic", except for 3 that don't help us that are in JS.
Their code examples in their docs (not the actual API docs) contain snippets from either aws native or their "crosswalk" product.
Crosswalk is their "better api" in that it does  more stuff for you, but obscures other stuff. They've chosen to fold crosswalk into their "language apis" but the status of that work is unclear.
There are more code examples in the docs in js.
"AWS native" is their 1 for 1 match between AWS and their language APIs but not all stuff has been implemented. For example I don't see aSubnet class. I noticed a code example that mixes both classic and aws native because of this.

In any case these are just observations. I think if we want to move forward with "classic" we may need to abandon the private subnets idea with pulumi. I could attempt to re-implement everything in "native". I've tried mixing native and classic without success. Tried passing the private subnet ids to the NodeGroup thinking it would deploy to them instead, but that just puts me back to the earlier problem of the nodes not being able to join the cluster.
Basically my takeaway is something like this: dividing a vpc up into public and private subnets was a requirement that the opted to implement in one of their newer apis, but sadly we don't have as good of a code example for that. Could be smooth sailing but I'm already seeing weirdnesses like there not being a Subnet class in aws native.

Wanted to link to the crosswalk stuff and language support. It really seems like that part of their codebase is where the kinds of benefits that we're looking for from pulumi live.

Superseded by https://github.com/culibraries/folio/pull/90


How do I add description in model then show in sawgger?

I hope to look at the description in sawgger (like picture).
How can I do write struct?
type ExamplePost struct {
Name string binding:"required" example:"jim"
Old int binding:"required" example:"30" format:"int"
}

It is actually possible but is not easy to figure out because is not documented.  :)
type Example struct {
// Comment for Message field
Message      string   json:"message" example:"Test message goes here ..."
}

Welcome PR to improve the swag’s documentation.

@JimChung0403 I wonder that if you have any solution for this question?

Is it possible to support trailing comment as description.
type Example struct{
    Message string `json:"msg"` // I get use to comment here
}

I think you should do comment like this:
type Example struct{
    // I get use to comment here
    Message string `json:"msg"`
}

Hi @xiaoguiSniper, I see how your codes work. But I prefer tailing comments especially on large struct, which looks good for me.
type LargeStruct struct {
    // comment i
    I string
    // comment prefer
    Prefer int
    // comment Tailing
    Trailing string
    // comment comments
    Comments string
    // comment though
    Though string
}

type LargeStruct struct {
    I        string // comment i
    Prefer   int    // comment prefer
    Trailing string // comment Tailing
    Comments string // comment comments
    Though   string // comment though
}


KAFKA-3522: TopologyTestDriver should only return custom stores via untyped getStateStore() method

Call for review @guozhangwang @bbejeck @vvcephei @ableegoldman @cadonna @abbccdda

Java8 failed. Java11 passed. Retest this please.

waiting on green jenkins to merge

Merged #6756 to trunk


Add word Median

The Pull Request resolves Issue #407
Description:
Add word Median

Hey @isharadilshan
Your filename doesn't match with the word included in the file. Update the file name or the file contents.
Thanks for contributing


returned error 9002 (TiKV server timeout)  when inject one tikv down while running sysbench oltp_read_only

Bug Report

What version of TiKV are you using?

What operating system and CPU are you using?

Steps to reproduce
NS：endless-oltp-tps-720692-1-766
1、[2022/03/18 07:54:55.877 +08:00] [INFO] [cmd.go:124] ["Start remote command"] [cmd="sysbench --mysql-user=root --db-driver=mysql --time=0 --report-interval=10 --mysql-db=test --mysql-host=tc-tidb.endless-oltp-tps-720692-1-766 --mysql-port=4000 --threads=64 --tables=32 --table-size=10000000 --mysql-ignore-errors=2013,1213,1105,1205,8022,8027,8028,9004,9007,1062 --rand-type=uniform oltp_read_only run"] [nodename=benchtoolset]、
2、[2022/03/18 08:05:56.098 +08:00] [INFO] [chaos.go:86] ["Run chaos"] [name=failure] [selectors="[endless-oltp-tps-720692-1-766/tc-tikv-3]"] [experiment="{"Duration":"","Scheduler":null}"]
What did you expect?
tidb client should not report error 9002 (TiKV server timeout)
What did happened?
returned error 9002 (TiKV server timeout)
[2022/03/18 08:06:36.329 +08:00] [INFO] [workload.go:471] ["FATAL: mysql_stmt_execute() returned error 9002 (TiKV server timeout) for query 'SELECT c FROM sbtest2 WHERE id=?'\r\n"]
logs：http://minio.pingcap.net:42041/buckets/test-infra-testground/browse/YXJjaGl2ZS9lbmRsZXNzLW9sdHAtdHBzLTcyMDY5Mi0xLTc2Ni8=

/type bug
/severity major
/assign sticnarf

/found automation

/assign NingLin-P

from NingLin-P：
This should be regarded as an expected problem. Each voter will randomly select a value from the range of 10s-20s as the election timeout. When no leader message is received after this time, it will initiate an election to become a candidate, and the candidate will vote for itself first and then to the candidate. Other voters request votes. In the case of 3 copies, if the election timeout of two voters is the same and they become candidates at the same time, then whoever's requestVote msg reaches the last voter first can get the vote and become the leader, but cannot vote because of tikv-3 injection failure. Since each candidate has already voted for himself, he will not vote for the other party (it can be voted in the pre-vote stage, but can be ignored for the time being), so it cannot become a leader. Then wait until the next election timeout, because the two voters randomly select the same election timeout, so the above situation occurs again, until tikv-2 initiates an election ahead of tikv-1 for the third time to become the leader.
timeline:
05:56 tikv-3 goes offline, the leader of 11635 is on it
06:05 tikv-1 & tikv-2 initiate election at the same time, split vote cannot elect leader
06:24 tikv-1 & tikv-2 initiate election at the same time, split vote cannot elect leader
06:42 tikv-2 initiates an election to become the leader

@sticnarf  can this issue be improved from tidb，such as delay backoff time ？

/remove-type bug

/remove-severity major


Code view scrolls when code lens is dismissed

VSCode Version: 1.9.1
OS Version: macOS Sierra 10.12.3

Steps to Reproduce:

Enable a code lens
Scroll down to a position where there is a code lens above the rendered lines
Disable the code lens

Result:
The code scrolls up a couple lines.
Expected:
Center line should stay centered.
I'm not sure this ever happens in VSCode proper, but embedding it on sourcegraph.com exhibits the issue. If you go here: https://sourcegraph.com/github.com/gorilla/mux/-/blob/mux.go#L52:74 and click the Show authors button a couple times it exhibited the issue before b86e32169ef2523826486cb7cf915d9cf9117cf1 was applied. That patch might be a starting point, but it probably needs additional checks.

Reproducible when toggling editor.codeLens from true to false.


[modify] stylelint on github actions

Coverage decreased (-0.03%) to 84.737% when pulling ce7006ae8a41db6c5f6b460ec0e8fb4c07d4b043 on pr-stylelint-on-github-workflows into 7c7e39cd85cff3f15d38ef9065ec5f204aa476b1 on master.


Generate builds via Makefile

Makefile is staged but needs to generate builds based on platform.

I am not sure what you are looking for here?  Or do you want make build to build the top few OSs and stick them in a directory?

I don't think you need this if all you want to do is make releases and publish them (#3).

Or do you want make build to build the top few OSs and stick them in a directory?

This, exactly! Thanks for the help @dixonwille.
I'm mostly trying to automate cross-compile for each release here. So being able to make build to a builds/ is the goal.

Resolved my #13


Explore code commenting support in diff editor for allowing a 'Review Github PR' extension

[ ] determine what this means

AFAIK, @alexandrudima doesn't have time on this feature this iteration. Remove the milestone first for better endgame query.


Fix #1618: Fix bronze table query error

Fixes #1618

Found the issue. The problem was that we were constantly nuking the bronze table folder and re-exporting the parquet file and because the lake ETL runs each few seconds the file was constantly deleted and re-created which caused the reading error. Modified to only nuke it after the seconds_between_export have passed from last file update


chore(*): upgrade to gradle 7.0

Closing this pull request since there has been no update from author of the repo, For folks landing on this PR with the same issue of migrating to gradle 7.0, Kindly use the below plugin
https://plugins.gradle.org/plugin/io.github.humblerookie.gradle


rootless: allow single ID mappings

we were playing safe and not allowed any container to have less than 65536 mappings.  There are a couple of reasons to change it:

it blocked libpod to work in an environment where newuidmap/newgidmap are not available, or not configured.

not allowed to use different partitions of subuids, where each user has less than 65536 ids available.

Hopefully this change in containers/storage:
https://github.com/containers/storage/pull/303
will make error clearers if there are not enough IDs for the image that is being used.
Closes: https://github.com/containers/libpod/issues/1651
Signed-off-by: Giuseppe Scrivano<EMAIL_ADDRESS>

One question for cosideration, otherwise LGTM

@jamescassell does the PR work for you?
@TomSweeneyRedHat Pushed a new version with the Debugf statement

LGTM

/lgtm


PDF Render Error

Attach (recommended) or Link to PDF file here:
Configuration:

Web browser and its version: FireFox 74.0
Operating system and its version: MacOS 10.14.6
PDF.js version: 2.3.200
Is a browser extension: No

Steps to reproduce the problem:

Use sample.pdf

upload to https://mozilla.github.io/pdf.js/web/viewer.html

What is the expected behavior? (add screenshot)

What went wrong? (add screenshot)

Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension): No

I can reproduce this using Arch Linux and Firefox. The following messages show up in the console:
Warning: getOperatorList - ignoring errors during "GetOperatorList: page 0" task: "RangeError: 14737632 is not a valid code point".
Warning: getTextContent - ignoring errors during "GetTextContent: page 0" task: "RangeError: 14737632 is not a valid code point".

This link is OK
http://<IP_ADDRESS>:8877/4d1091cde93c899/web/viewer.html
Thanks a lot


Falling into Crawl Traps

Hi,
To start, thank you for an excellent piece of work.  Appreciated.
I'm trying to use this to crawl the site http://www.udemy.com/ . I added it to my Gemfile, bundle install and started it up and everything looked great.  What I found was that it fell victim to crawl traps generating urls like this:
https://www.udemy.com/courses/photography/mobile-photography/all-courses/?p=324
The actual number pages on the base url:
https://www.udemy.com/courses/photography/mobile-photography/all-courses/
is only 3 so its very much spidering far, far deeper than needed.
Any suggestions for how to go about addressing this?
What I'm trying to do is build a page_archiver and my core loop looks like this (its being executed from a Rake task):
statistics = CobwebCrawler.new(:cache => 600, :thread_count => 10, :valid_mime_types => ["text/html"]).crawl("http://www.udemy.com") do |page|
  puts "Just crawled #{page[:url]} and got a status of #{page[:status_code]}."
  if page[:mime_type] == "text/html"
    page_ctr = page_ctr + 1
    #puts page.title
    #debugger
    page_archive = PageArchive.find_or_create(page[:body], page[:url])
    total_time = (Time.now - start_time)
    puts "  Total time: #{total_time}"
    puts "  Total pages: #{page_ctr}"
    puts "  Time per page: #{total_time.to_f / page_ctr}"
  else
    puts "  Not text/html for: #{page[:url]}"
  end

end

After running it for about 20 minutes, it got 10,000 "pages" deep almost all of which was just "psuedo pages" like the ?p=324 url.
I didn't see any kind of configuration option that would limit progress so this feels like something internal to the guts of the crawler but if I've missed something, my bad.
Thanks
Scott

As a quick solution you could exclude the page 4 url?  I'll look at it in more detail when i get back home.

Yeah, if you add to your external_urls config option the url for page 4 then it should exclude that page and assuming the subsequent pages are only crawled because its gone into page 4 then it will stop at that point.  A better solution would be giving the ability to prevent the subsequent crawl of links within the current page as you are processing it.  This would mean you can detect this issue, maybe due to no results in the page, and mark it as not to be processed.

Hi Stewart,
I'm more than willing to take a stab at fixing this.  Mind giving me a pointer as to where to best start so I don't make a hash of your nice work?

I think that possibly the best solution would be to include the internal_links data in the hash passed to the block, that way in your code you can add and remove items to and from the list, giving you control over what next steps the crawler will take.  That would mean moving the yield above the internal_links.each call on line 121, and changing "internal_links.each" to "content[:internal_links].each" so that updates within the block are now used.
So in your scenario, you would detect there are no items returned in the page body and remove the "?p=" link.
Thats my thinking just now, haven't had a chance to try it out though yet.

  I was about to tell you that this was a Cobweb level issue -- but its not.  The site itself is buggy:
https://www.udemy.com/courses/photography/mobile-photography/all-courses/
and it has a link to:

So you goto https://www.udemy.com/courses/photography/mobile-photography/all-courses/?p=2"
and it has a link to:

So you goto  https://www.udemy.com/courses/photography/mobile-photography/all-courses/?p=3
and it has a link to:

which doesn't exist -- YEP -- this is a site level bug and it HAS to be handled at my specific application logic.  The site is generating an infinite succession of pagination offsets even though the there are only 3 pages.  Sheesh.    Apologies.
The only question becomes how does this get handled at a extensible approach without my having to maintain my own fork -- and I don't have a great answer for this.  The only thing that comes to mind is some kind of is_link_valid? method but given that this is javascript pagination, even knowing that the link is invalid is hard.
Kudos by the way for supporting  . I don't think I've ever put that into any crawlers I wrote from scratch.
The only question becomes how does this get handled at a extensible approach without my having to maintain my own fork -- and I don't have a great answer for this.  The only thing that comes to mind is some kind of is_link_valid? method.  Another possibility is maybe not following <link tags at all -- but then I see no way to navigate this particular succession of content.
I was able to confirm that one other gem, spidr, has this exact same problem which doubly confirms that the site is actually buggy.  Interestingly Google doesn't have this issue - perhaps it is because they simply aren't indexing the pagination but that's odd for Google.
This is really a problem of identifying duplicate content.  An easy way to solve this might be to look at a signature on just the html from the <body tag forward.  I took two of these pages that were technically invalid -- the ?p=4 and ?p=5 and did a wget on them.  Then I diff'ed them and the only difference was the single <link tag that was invalid.  Then I noticed that the <link tag was in the  element.  So I removed the html up to the <body and diff'ed them again and at that point they were the same page i.e. duplicate content.
So one possible approach might be to keep an SHA hash of the content from <body forward and compare to see if this has already been processed -- or matched the last thing processed (much, much harder in a threaded crawler; been there; fought that battle).
Thoughts?  I can certainly hack in fix for my own needs but it likely means maintaining a fork indefinitely which always sucks and identifying duplicate content is really a core crawler issue.  Let me know if you're interested in this.  I really like cobweb and its the first open source crawler I've found that really meets my needs so I'm willing to help get this addressed if you're interested in the help.
Thank you so much.

Hey Stewart -- any thoughts about what I wrote up above?  I haven't forked anything yet and I'd really prefer not to because I truly think that this belongs in the core.

There isn't any issue with forking, it's the normal process for adding to open source projects.  If you fork, you can work on adding functionality, documentation or anything within the codebase.  Make sure you work in a feature branch and then when you are ready, you can request a pull request into the core repository, we can then work on if anything needs to change and then it gets merged into core.
I'll have a quick go at it today, and let you know how it goes, otherwise, feel free to fork and make changes, as long as it is in a branch its easy to merge back into core.

I think there is properly done forking -- and you're correct about that -- and there is forking where your goal is to fix something for your own needs.  My concern is that this feels to me like a core issue that needs to get addressed by enhancing the core structure and if someone unfamiliar with the code does it (me), they're likely to screw it up badly.

Just to let you know Stewart I am tackling this and I think I have a fairly elegant approach (at least in my head; implementation is where it gets icky).

On my fork I've placed a wiki entry with the start of my proposed change: https://github.com/fuzzygroup/cobweb/wiki/Duplicate-Content-Detection


Two consecutive stackTrace requests when a breakpoint is hit.

Version 1.23.1
Commit d0182c3417d225529c6d5ad24b7572815d0de9ac
Date 2018-05-10T17:11:17.614Z
Shell 1.7.12
Renderer 58.0.3029.110
Node 7.9.0
Architecture x64
Steps to Reproduce:

Install java debug extension published by Microsoft, add the following settings:
"java.trace.server": "verbose",
"java.debug.logLevel": "verbose"

Create a java file, set a breakpoint and F5, choose java.

/**
 * Hello world!
 *
 */
public class App
{
    public static void main( String[] args )
    {
        System.out.println( "Hello World!" );
    }
}

Wait until breakpoint is hit
Open command panel by Ctrl+shift+p, input "Open Java language Server log file"

{"event":"breakpoint","body":{"reason":"new","breakpoint":{"id":1,"verified":true,"line":11,"message":""},"type":"breakpoint"},"seq":9,"type":"event"}

!ENTRY java-debug 1 0 2018-05-11 14:29:31.844
!MESSAGE
JDI Event: BreakpointEvent: 7

!ENTRY java-debug 1 0 2018-05-11 14:29:31.849
!MESSAGE
[[EVENT]]
Content-Length: 130

{"event":"stopped","body":{"threadId":1,"reason":"breakpoint","allThreadsStopped":false,"type":"stopped"},"seq":10,"type":"event"}

!ENTRY java-debug 1 0 2018-05-11 14:29:31.855
!MESSAGE
[request]
{"command":"threads","type":"request","seq":7}

!ENTRY java-debug 1 0 2018-05-11 14:29:31.863
!MESSAGE
[[RESPONSE]]
Content-Length: 302

{"success":true,"request_seq":7,"command":"threads","body":{"threads":[{"id":2,"name":"Thread [Attach Listener]"},{"id":3,"name":"Thread [Signal Dispatcher]"},{"id":4,"name":"Thread [Finalizer]"},{"id":5,"name":"Thread [Reference Handler]"},{"id":1,"name":"Thread [main]"}]},"seq":11,"type":"response"}

!ENTRY java-debug 1 0 2018-05-11 14:29:31.869
!MESSAGE
[request]
{"command":"stackTrace","arguments":{"threadId":1,"startFrame":0,"levels":20},"type":"request","seq":8}

!ENTRY java-debug 1 0 2018-05-11 14:29:31.916
!MESSAGE
[request]
{"command":"threads","type":"request","seq":9}

!ENTRY java-debug 1 0 2018-05-11 14:29:31.926
!MESSAGE
[request]
{"command":"stackTrace","arguments":{"threadId":1,"startFrame":0,"levels":20},"type":"request","seq":10}

!ENTRY java-debug 1 0 2018-05-11 14:29:32.117
!MESSAGE
[[RESPONSE]]
Content-Length: 311

The log shows vscode has sent two same stackTrace requests, while in previous versions, sometimes only one stackTrace is sent and sometimes two stackTrace events are sent, this bug also occurs on latest insider version.
Does this issue occur when all extensions are disabled?: Yes

@andxu thanks for the details.
And the expected behavior is that only one stackTrace request is sent?
Your debug adapter does not supportsDelayedStackTraceLoading correct?
If you F1 > developer tools > put a breakpoint here
Who are the two callers of the stackTrace request?

@andxu thanks for providing more details. I believe the second call is fine, since that is being called because of a stop event.
Let's investigate the first one a bit more, what caused that.
It seems like the callStackChangeScheduler got scheduled, we need to find a reason why. Can you put a breakpoint here and figure out the reason for this.
My assumption is that this might be by design, more preciesly when a new debug session starts vscode sends out a threads request to get all the inital threads, when that is done vscode will ask for the children of a thread if you report that some of those threads are stopped - which I believe your adapter does - and that might be the issue.
Independent of that a stop event comes and vscode sends a stackTrace request which you see as the second request.


feat: add type to status -> HTTPStatus

Hi !!!
I'm testing this framework that seems great to me!
Doing some testing, it occurred to me to add some improvement to the set.status. To avoid mistakes.
For example
app.post("/test", ({ body, set }) => {
   set.status = 100
   return body
})

It gives this error:
The status provided (100) must be 101 or in the range of [200, 599]
I came up with this solution:
	// 2xx: Success - The action was successfully received, understood, and accepted
	200 // OK
	| 201 // Created
	| 202 // Accepted
	| 203 // Non-Authoritative Information
	| 204 // No Content
	| 205 // Reset Content
	| 206 // Partial Content
	| 207 // Multi-Status
	| 208 // Already Reported
	| 226 // IM Used

	// 3xx: Redirection - Further action must be taken in order to complete the request
	| 300 // Multiple Choices
	| 301 // Moved Permanently
	| 302 // Found
	| 303 // See Other
	| 304 // Not Modified
	| 305 // Use Proxy
	// | 306 // (Unused)
	| 307 // Temporary Redirect
	| 308 // Permanent Redirect

	// 4xx: Client Error - The request contains bad syntax or cannot be fulfilled
	| 400 // Bad Request
	| 401 // Unauthorized
	| 402 // Payment Required
	| 403 // Forbidden
	| 404 // Not Found
	| 405 // Method Not Allowed
	| 406 // Not Acceptable
	| 407 // Proxy Authentication Required
	| 408 // Request Timeout
	| 409 // Conflict
	| 410 // Gone
	| 411 // Length Required
	| 412 // Precondition Failed
	| 413 // Content Too Large
	| 414 // URI Too Long
	| 415 // Unsupported Media Type
	| 416 // Range Not Satisfiable
	| 417 // Expectation Failed
	// | 418 // (Unused)
	| 421 // Misdirected Request
	| 422 // Unprocessable Content
	| 423 // Locked
	| 424 // Failed Dependency
	| 425 // Too Early
	| 426 // Upgrade Required
	| 427 // Unassigned
	| 428 // Precondition Required
	| 429 // Too Many Requests
	| 430 // Unassigned
	| 431 // Request Header Fields Too Large
	| 451 // Unavailable For Legal Reasons

	//  5xx: Server Error - The server failed to fulfill an apparently valid request
	| 500 // Internal Server Error
	| 501 // Not Implemented
	| 502 // Bad Gateway
	| 503 // Service Unavailable
	| 504 // Gateway Timeout
	| 505 // HTTP Version Not Supported
	| 506 // Variant Also Negotiates
	| 507 // Insufficient Storage
	| 508 // Loop Detected
	| 509 // Unassigned
	// | 510 // Not Extended (OBSOLETED)
	| 511 // Network Authentication Required

Reference link:
https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
Thank you very much for creating this framework!!
I am at your disposal if any changes, improvements, etc. need to be made.

Based on your PR, this feature is now implemented into 0.7 release.
Thanks for your contribution.


Remove now redundant sckitlearn hack

Checked check_registry works by first inspecting draft PR directly onto master.

@OkonSamuel Friendly reminder that a review would be nice.

I'm happy to drop it. But I'm trying to remember why we had it in the first place


Integration tests stall (logging becomes unreliable)

Continuation of https://github.com/microsoft/vscode/issues/140572
On Linux, remote integration tests timed out for https://dev.azure.com/monacotools/Monaco/_build/results?buildId=152671 :
2022-01-28T14:40:00.6589879Z Found --crash-reporter-directory argument. Setting crashDumps directory to be '/__w/1/s/.build/crashes'
2022-01-28T14:40:00.6601746Z [19554:0128/144000.659642:ERROR:appcenter_api.cc(55)] expecting appcenter url prefix
2022-01-28T14:40:00.7324654Z [19589:0128/144000.731641:WARNING:vaapi_wrapper.cc(588)] VAAPI video acceleration not available for swiftshader
2022-01-28T14:40:00.7328806Z [19589:0128/144000.732325:ERROR:gpu_init.cc(440)] Passthrough is not supported, GL is swiftshader
2022-01-28T14:40:00.9858696Z [main 2022-01-28T14:40:00.984Z] update#ctor - updates are disabled by the environment
2022-01-28T14:40:01.0457368Z [main 2022-01-28T14:40:01.044Z] ExtensionHostStarterWorker created
2022-01-28T14:57:02.5569436Z ##[error]The task has timed out.
2022-01-28T14:57:02.5580918Z ##[section]Finishing: Run integration tests (Remote)

After analyzing the logs, here are my findings:

the extension host is started but the log statement immediately after the fork doesn't appear in main.log: https://github.com/microsoft/vscode/blob/22fc0438ffad90f2ee34d4b0c46df61d3222afe7/src/vs/platform/extensions/node/extensionHostStarterWorker.ts#L51-L60

[2022-01-28 14:40:00.985] [main] [info] update#ctor - updates are disabled by the environment
[2022-01-28 14:40:01.045] [main] [info] ExtensionHostStarterWorker created

$getCanonicalURI is invoked by the renderer on the extension host (appears in exthost.log:

[2022-01-28 14:40:03.443] [exthost] [info] Extension host with pid 19652 started
[2022-01-28 14:40:03.443] [exthost] [info] Skipping acquiring lock for /tmp/tmp.EYOENEWPS8/User/workspaceStorage/18977c2027ffaf373bc3a4066c620658.
[2022-01-28 14:40:03.477] [exthost] [info] $getCanonicalURI invoked for authority (test)
[2022-01-28 14:40:03.482] [exthost] [info] ExtensionService#_doActivateExtension vscode.vscode-test-resolver, startup: false, activationEvent: 'onResolveRemoteAuthority:test'

IExtensionHostStarter.start doesn't return from the main process to the renderer because this log statement is missing: https://github.com/microsoft/vscode/blob/22fc0438ffad90f2ee34d4b0c46df61d3222afe7/src/vs/workbench/services/extensions/electron-browser/localProcessExtensionHost.ts#L463-L469
I can add more logging from the renderer right before it invokes $getCanonicalURI, but to me it looks like the main process is no longer healthy, i.e. it doesn't print the log the extension host worker asks it to print and it doesn't return the start call to the renderer process.

@deepak1556 might need your advice here: our integration tests bring up a VSCode window, same as the smoke tests do. I see some extra arguments that we only use for smoke tests on Linux:
--electronArgs="--disable-dev-shm-usage --use-gl=swiftshader"

Do you remember why we use those? Would it make sense to use the same, and should we maybe also disable GPU?

Oh interesting, nevermind, we set the same flags for integration tests too, but from within the shell script.

Bizarre indeed.
Logs are not helpful because we try to be less spammy, the only hint is
update#ctor - updates are disabled by the environment

that the main process has started for the fifth run but stalls after
ExtensionHostStarterWorker created

We do not even know if a window has opened or not. Though probably not because a typical message we get is this one:
[DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues.

So maybe the window that wants to open is broken and not the main process?

I think the window managed to start up because it called back into the main process and asked it to start an extension host process. It also called $getCanonicalURI on the extension host process. But nothing from the window appears in renderer1.log. It should print IExtensionHostStarter.start() took XX ms. after the main process returns from the start() method.
I can add even more logging in the extension service on the renderer using the condition if (env['CI'] || env['BUILD_ARTIFACTSTAGINGDIRECTORY']) to not pollute our regular runs, but I'm really not sure what's going on.

Someone in the past added this code (not sure, maybe @joaomoreno)
kill_app() { killall $INTEGRATION_TEST_APP_NAME || true; }

This will terminate VSCode, but maybe not the remote server before the next test run. But it seems to me that the remote server couldn't cause the issues we are seeing, because the window should still be able to log messages no matter what.

@alexdima this failed again in here:
### Emmet tests

Found --crash-reporter-directory argument. Setting crashDumps directory to be '/__w/1/s/.build/crashes'
[9790:0201/222705.226755:ERROR:appcenter_api.cc(55)] expecting appcenter url prefix
[9820:0201/222705.286413:WARNING:vaapi_wrapper.cc(588)] VAAPI video acceleration not available for swiftshader
[9820:0201/222705.287095:ERROR:gpu_init.cc(440)] Passthrough is not supported, GL is swiftshader
[main 2022-02-01T22:27:05.480Z] window#load: attempt to load window (id: 1)
[main 2022-02-01T22:27:05.492Z] update#ctor - updates are disabled by the environment
[main 2022-02-01T22:27:05.556Z] ExtensionHostStarterWorker created
[main 2022-02-01T22:27:06.736Z] window#load: window reported ready (id: 1)
Invoking getCanonicalURI for authority test...
getCanonicalURI returned for authority test.
Starting to wait on IWorkspaceTrustManagementService.workspaceResolved...
##[error]The task has timed out.
Finishing: Run integration tests (Remote)

Is this maybe the culprit: Starting to wait on IWorkspaceTrustManagementService.workspaceResolved ?

Yes, it looks like that promise never returns:
https://github.com/microsoft/vscode/blob/482daf0366fcde6e079826df895566987af8f547/src/vs/workbench/services/extensions/electron-browser/extensionService.ts#L392-L403

Seems to be originating from here:
https://github.com/microsoft/vscode/blob/4f4df625f901f020606e580bda55e82d31ebaed9/src/vs/workbench/services/workspaces/common/workspaceTrust.ts#L149-L159

This is really not my area of expertise, let me know if you need help. Please investigate.

@lszomoru The problem is that remote integration tests stall. It is not clear why. So far, by adding more logging we have tracked it down to the await IWorkspaceTrustManagementService.workspaceResolved:
https://github.com/microsoft/vscode/blob/482daf0366fcde6e079826df895566987af8f547/src/vs/workbench/services/extensions/electron-browser/extensionService.ts#L392-L403
I know that the workspace trust service ends up invoking getCanonicalURI so I also added logging there to rule that out as a reason for a stall:
https://github.com/microsoft/vscode/blob/cab9713422fed8d1a1c052a18cf2215a6d80bec4/src/vs/workbench/services/extensions/electron-browser/extensionService.ts#L380-L389
The logs contain:
[2022-02-01 22:27:07.004] [renderer1] [info] Invoking getCanonicalURI for authority test...
[2022-02-01 22:27:07.006] [renderer1] [info] getCanonicalURI returned for authority test.
[2022-02-01 22:27:07.007] [renderer1] [info] Starting to wait on IWorkspaceTrustManagementService.workspaceResolved...

It appears that there is a stall in IWorkspaceTrustManagementService.workspaceResolved, but which is not caused by the getCanonicalURI invocation.

I will try to investigate the issue, though I though that we have decided to disable workspace trust in our test runs. Looking at the file here I see that we are using the --disable-workspace-trust flag so maybe I have missed some location. It's not clear to me when workspace trust code is being executed to begin with.

Again here:
https://dev.azure.com/monacotools/Monaco/_build/results?buildId=154352&view=logs&j=963754b8-394e-5caf-de5d-2790b068caec&t=6b00722f-1101-5770-9e0c-f427af83d823
-- friendly neighborhood build champion

@lszomoru can we unblock the CI and skip this line when workspace trust is disabled:
https://github.com/microsoft/vscode/blob/4a130c40ed876644ed8af2943809d08221375408/src/vs/workbench/services/extensions/electron-browser/extensionService.ts#L399

Sorry, will look into this issue first thing tomorrow and will try to come up with a solution.

Not to my knowledge, especially since the test config is the same for macOS and Linux. There is a few electron related arguments only on Linux though:
https://github.com/microsoft/vscode/blob/01df559c184549248fcfbcecc5a616462e00c252/scripts/test-remote-integration.sh#L11

Not really news at this point but just doing my job :)
https://dev.azure.com/monacotools/Monaco/_build/results?buildId=155005&view=logs&j=963754b8-394e-5caf-de5d-2790b068caec&t=6b00722f-1101-5770-9e0c-f427af83d823&l=11176

@bpasero and I have spoken today and I have added more logging to the code inside the workspace trust service. Hopefully those logs will allow us to track down the root cause of the hang. We have looked at the historical failures and what seems to be interesting interesting, that the first test suite always succeeds while the second suite hangs. Will keep an eye on the build and review the logs in case we get another failure.

@lszomoru it looks like we have a build with your new logging that stalled:
https://dev.azure.com/monacotools/Monaco/_build/results?buildId=155814&view=logs&j=963754b8-394e-5caf-de5d-2790b068caec&t=a5663145-e9fb-5b6a-6210-7b66b21662a2&l=15650
I think the last entry I see is:
[WT] Open workspaceResolved gate...
[WT] Open workspaceTrustInitialized gate...

Does this help?

This looks like a case in which the integration tests have timed out, and not the remote integration tests. I have checked the logs and from what I can tell, the hang is not related to the workspace trust code, as everything seemed to have executed correctly.

Oh yeah sorry, this was not even a remote test, I should have payed more attention.

Somehow this does not seem to happen anymore over 7 days 🤷

I suggest to close this and reopen if we have more data. Maybe my fixes to native-keymap have stabilized this or the update to Electron 17. But not seeing the hangs is a good sign, let's leave all the logging in place and reopen when we see these hangs again.


Libs.h function to ask the user for values

{
  selector: "some field",
  value: Libs.h.question("What to insert here?")
}

Opens a dialog (aka. prompt())

Too edge casey :)


Operator adds (hard-coded?) sha256 to any ArgoCD image in Deployments

Environment:

OpenShift 4.6.16
ArgoCD Operator 0.0.14 (installed via OperatorHub)
Artifactory for DockerHub pull-through mirroring

I want to add the helm-sops plugin to my ArgoCD instance that I'm managing with the ArgoCD operator. In order to do so, I'm building my own derived ArgoCD image.
When trying to use my own image (by specifying it in the ArgoCD CR) instead of the one from DockerHub, the ArgoCD operator appends a (hard-coded) sha256 hash to the image string in each of the relevant Deployments (application-controller, repo-server and server). This hash causes pulling the image to fail with "name not known to the manifest" because the hash is obviously not matching with the hash of my image.
When I'm shutting down the operator (i.e. scaling it to 0) and remove the sha256 from the Deployments manually, the image can be pulled immediately.
I guess, the ArgoCD operator is retrieving the hash when the ArgoCD instance is initially deployed and sticks with it. Can you please change this behavior so that it is possible to change the ArgoCD image at any time?
Edit: I just tried to completely delete and recreate my ArgoCD instance and the operator shows the same behavior in that case as well.

Nevermind: I completely overlooked that I have to explicitly specify the "version" property instead of adding the version tag to the image string.

@Obirah  As I remember with the operator 0.14 there is this problem, even going to specify the image tag
Hopefully I ask who knows best if he is still so 'or if we have something wrong in cr definition.

It worked with specifying the version in the ArgoCD's spec. It's just important to delete and fully recreate the ArgoCD because it looks like the operator cannot cover every edge-case of reconciliation. (Similar to this issue, I had a few months ago: https://github.com/argoproj-labs/argocd-operator/issues/211 )


*** The first time issue of ZXingScannerView on Android ***

The ZXingScannerView with ZXing.Net.Mobile.Forms and ZXing.Net.Mobile 2.4.1 has the first time issue on Android (the old version of ZXing such as 2.3.2 may not have this issue for the old version of Xamarin.Form). Install the app with this component, it can't scan  QR code at the first time. After quit the app or the page with ZXing, and enter the page with ZXingScannerView, the problem is solved. Please fix it. Thanks!

same here

Same for me, no problem on the iOS solution (Xamarin project)

Fixed in latest 3.0 series.


Custom vendor/product id?

I'm working with some CP210x devices with a custom vendor/product id. Can I use this library? The ids seem to be fixed in CP210xIds. In e.g. https://github.com/mik3y/usb-serial-for-android it's possible to pass a custom table (just mentioning it because I genuinely wonder why it's possible there and not here)

@i-schuetz  For any interested this will be solved in the next update.

@i-schuetz Use
UsbSerialDevice serial = UsbSerialDevice.createUsbSerialDevice(UsbSerialDevice.FTDI, device, connection,0);


Kyber and known answer tests

I tested the Kyber Cryptol spec for decryption using the known answer tests from NIST.  To do it, I created an AWK script that reads the NIST responses and generates a Cryptol batch file.  Submitting that file to Cryptol shows that for every one of the 100 known answer tests, the answers disagree.
There are three attachments.  The first one is the AWK script (you have to remove the .txt extension).  Second one is a bash script that sets the CRYPTOLPATH (you have to remove the .txt extension and make it executable). The final file is a transcript that shows how to create the batch file and the results of running it.
kat.awk.txt
cryptol-specs.txt
kyber.txt

Assigned to @mariosge and @bboston7 just for visibility. We'll triage soon.
Thanks @ramsdell for submitting the issue!

Try generating the batch file using the attached version of the kat AWK script.  The batch file generated makes the answers match.  I think there is something wrong with BytesToBits.
katrev.awk.txt

Thank you for submitting this issue @ramsdell ! Such review is super useful. I created a PR that solves it here:
https://github.com/GaloisInc/cryptol-specs/pull/58


!apex should show more useful information

Bug report: Still very generic embed, should be updated for more useful information
Submitted by: TheTestLOL#6274
Server: mfTesting

Marking as closed, due to arenas being avaialble in Apex, what else can really be shown?
This seems fine enough to me:


No cache option in dev branch

Hey there is no downscale or cache option in dev branch due to which I am unable to train the network. Could you guide how to add that please @kwea123

The training inputs are stored in
https://github.com/kwea123/nerf_pl/blob/19a290103fd8df211a85a150daff861b53d59942/datasets/llff.py#L244-L245
So you can save these tensors in files and read them without the need to recompute them in each training/on different GPUs. However, this should not be a problem making you unable to train, caching only reduces data I/O so the preprocessing runs faster.


C#: misuse of InvalidOperationException on network error

Consider
Channel channel = new Channel(machine, ChannelCredentials.Insecure);
var client = new Compiler.CompilerClient(channel);
using (var call = client.Compile())
  call.RequestStream.WriteAsync(new CompileRequest()).Wait();

(the .Wait() here is abused for demo purposes only)
If machine is not listening, them the WriteAsync call will throw InvalidOperationException with the message Send failed. There is a huge problem with that.

[TL;DR: gRPC is using a wrong exception class].  InvalidOperationException semantics is quite close to that of C++ ASSERT; it says, essentially, "you are doing something wrong." There is no explicit guideline published by MS as far as I know (and, unlike Win32, I cannot commend .NET docs on exhaustiveness), but the documentation opposes the IOE to the ArgumentException. Other development community guidelines (a good example) talk about NotSupportedException as opposed to NotImplementedException, and there is also of course ObjectDisposedException. What is common among these presumed opposites is that all of them tell the programmer that the code is not using the object correctly. Kind of what the ASSERT does.

Now this is clearly not our case. I the programmer cannot control the state of network! InvalidOperationException is out of place here. I cannot think of a good predefined class for an exception in this case. For one, RpcException with the Unavailable status would certainly not be out of place here (and I think the previous gRPC did do just that, in v0.11).

[TL;DR: gRPC is throwing out information about error condition]. Consider

try {
  . . .
} catch (InvalidOperationException ex) {
  if (ex.Message == "Send failed") {
    . . .

I almost hear you saying ugh! as in ugly. Unfortunately, the exception message is the only additional piece of information that  I have here. And, the scariest thing, this is how my production code currently looks. Now, using another exception type would be a great help to avoid this ugliness. I am talking only an enhancement here, so this is a lesser point. In the debug log there is (with long paths snipped for clarity):
I0701 17:25:24.184428 0 c:\jenkins\[snip]\subchannel.c:642: Connect failed:
 {"created":"@1467419124.184000000","description":"OS Error","file":"c:\jenkins\[snip]\tcp_client_windows.c","file_line":114,"os_error":"No connection could be made because the target machine actively refused it.\r\n","syscall":"ConnectEx","wsa_error":10061}

Would not I love to have access to this information through exception properties! Even in a form of loosely defined key/value pairs. I can make decisions like whether I should complain about invalid configuration, or keep calm and retain the address in the load balancer list to retry later.

+1 to both points. I was left scratching my head as well and it didn't occur to me to dig into logging just to find out the why of Send failed. I just assumed I was doing something wrong (just starting out with grpc here).
RpcException indeed would be a good starting substitution here instead of IOE imho. If MSDN is any nudge to how MS tends to handle these case,  DirectoryException / CommunicationException are good examples of this sort of situation handling. They're all encompassing within their realm and throw more specialized exceptions where they make sense (for example LdapException or SecurityAccessDeniedException)

@kkm000, thanks for filing this issue, I fully agree with your thoughts. Seems like this is something we need to fix. Also, I apologize for not reacting sooner, I was out of office for extended period of time.
Here are my thoughts on what needs to be done and how. I will start with the client side calls

For unary calls, everything works as expected because if such call fails, one receives an RpcException with appropriate status and InvalidOperationException is only thrown if programmer breaks the contract and does something not admissible.

For streaming calls, I think for a failed write we should deliver an RpcException with a StatusCode corresponding to the status code of the entire call. As a principle, a single gRPC call should only have a single status code associated with it - so in C# API we need to honor that. Internally in C core, finishing of a write operation and finishing of the entire call with a status code are two different events. So, what needs to happen is when a write operation fails, we also wait for the entire call to finish and then throw RpcException with the correct status because we already obtained it. That should work fine because

C core guarantees that if a write operations fails, the entire call will fail "immediately" (soon enough) afterwards. So we wouldn't starve waiting on the status code to be delivered.
there should be no performance impact, because we only wait for two events instead of one in case the RPC call fails and that's not really something we need to optimize for.

Does that make sense? If so, things seem to be fairly clear on the client side.
On the server side, things works similarly with the biggest difference being that we cannot throw and RpcException (that comes with a status code) simply because server side calls don't have a status code associated with them - if they fail, they finish with a binary success/non-success flag reported back to the server code. So on the server side, we need to figure out which exception type is the most fitting here - my personal favorite seems to be IOException (but do be honest I am not 100% convinced I like that choice either).
What is your opinion on what the server side behavior should be (and what exception type to use)?

CC @apolcyn  @jskeet

Would it be worth it to try and do this in a non-breaking way, or at least less breaking? Possibly sub-classing the current InvalidOperationException? (got rid of one idea like that in https://github.com/grpc/grpc/pull/7861).

So what server-side failure type are we talking about? Something like "client killed connection"?

@jskeet  yes, if something went wrong with the call and we weren't able to finish the call normally (by sending client a status). That could be network error, client died, etc.

Sounds like IOException makes sense in that case then.

Another option would be to throw parameterless "RpcException" for more consistency with client side (but it's just an idea).

I like the idea of throwing RpcException, a single exception type to signify "this rpc has failed".
IOException feels a little out of place given that the grpc does not model IO, instead we model the invocation of methods that happen to require IO. It would be the only place in grpc that would throw an IOException requiring that exception handlers must now catch two types to handle the "this rpc has failed [and potentially retry]". RpcException seems semantically okay here.

Actually RpcException on server side turned out to be a bit tricky because it already has field Status that only makes sense on the client side. I had the options to introduce a new "ServerRpcException"  or stick  with "IOException".  It seems that IOException is the best choice, because it required the least changes and expresses the concept quite well. I didn't want to add just another exception type for no good reason.
See #8084 for the PR with the fix.

Sounds great. So looking briefly at #8084, we now throw an RpcException on the client, and an IOException on the server?

@imiuka Yes, exactly (once we merge and release of course).

#8084 has been been merged.


Use Latest Socket.io Client

Its a small change but will let anyone using the current socket.io ver 2.x work with this

@julianfrank thanks for your contribution!


release CI: Always use latest minor version of cibuildwheel

By removing the exact minor and patch version of the cibuildwheel action, it will always use the latest minor/patch version.
Major version updates will be handled by #168.

Thanks for the PR!
I am not sure why this didn't run any CI?

Not sure why this didn't run, changes to the release.yml file are specifically whitelisted for running CI:
  pull_request: # also build on PRs touching files that affect building sdist / wheels
    paths:
      - ".github/workflows/release.yml"

which I thought was working properly based on prior PRs...

But also the normal tests should in principle always run for any PR.

My mistake, cibuildwheel (currently) doesn't allow this and requires pinned major.minor.patch versions specified. That's also probably why the CI isn't running. So closing this PR.
#168 still does it job, and #170 can thus be merged.


Add Chinese translation

在翻译完提交之后才发现原来有人已经翻译了中文版本  #17 ，

在翻译完提交之后才发现原来有人已经翻译了中文版本 #17 ，

看到这个项目的第一想法是，看有没有中文版，没有？可以翻译。嗯，先看看PR中有没有在翻译的。果然有… 😆

在翻译完提交之后才发现原来有人已经翻译了中文版本 #17 ，

Hi, @OnePieceJoker, do you have a suggestion about what we can do about it? Since #17 is not yet merged, I would suggest you to comment there and suggest changes/sentences that you used in this PR, when applied. I'm suggesting this because other people already reviewed that PR.
@seriouszyx @geminiyellow I also appreciate if you could take a look and have any suggestions on resolving this.

@RomuloOliveira You are right, I will go to #17 to submit my comment.Thanks

@RomuloOliveira yep, the best solution is review #17 .
#17 need more reviewer, right?

Should I wait for some additional reviews before merging it?

so, @OnePieceJoker 实在抱歉，这个 PR 可以考虑并关掉它，然后在 #17 上进行修改。

@geminiyellow 我觉得在 #17 上提出更多的建议好一点，还是把这个PR关掉了


RFE - Enhance DO to prevent AWS hostname updates via dhclient

We are currently running into an issue in AWS where a DO specified hostname is not being maintained on a BIG-IP instance.  It appears that AWS is overwriting our device hostname via dhclient even though dhclient is disabled in the db settings.  It would be great if DO provided a configuration mechanism to prevent this situation.

I had this same issue before. I managed to get it working after fixing my DO json file.
Looks like it had a problem with creating an user object and it rolled back the rests of the DO configurations.
If you manage to GET ( https://<BIG-IP>/mgmt/shared/declarative-onboarding), you will see a "selfLink" line. Then you should try to GET that URL and see if there is any issue.
Mine looked like this:
{
    "id": "4fcdcc04-0c5c-442d-8e1f-dfe46e145705",
    "selfLink": "https://localhost/mgmt/shared/declarative-onboarding/task/4fcdcc04-0c5c-442d-8e1f-dfe46e145705",
    "result": {
        "class": "Result",
        "code": 200,
        "status": "OK",
        "message": "success"
    },

Of course, you need to change the localhost by your dns or bigip's name when using GET to check results.
Hope it helped!

@btully What DB settings are you trying to set?
I have created an internal product backlog item AT-807 to apply the work around found in K45728203 when a hostname is specified in a DO declaration.

This was resolved a few versions ago. Closing this issue. If you are still experiencing a problem with the latest version, please reopen the issue or create a new one.


[SPARK-12044] [SparkR] Fix usage of isnan, isNaN

1, Add isNaN to Column for SparkR. Column should has three related variable functions: isNaN, isNull, isNotNull.
2, Replace DataFrame.isNaN with DataFrame.isnan at SparkR side. Because DataFrame.isNaN has been deprecated and will be removed at Spark 2.0.
3, Add isnull to DataFrame for SparkR. DataFrame should has two related functions: isnan, isnull.
cc @shivaram @sun-rui @felixcheung

Test build #46874 has started for PR 10037 at commit 6805952.

Test build #46874 has finished for PR 10037 at commit 6805952.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46874/
Test PASSed.

LGTM except that it's good to add additional test cases.

Test build #47058 has started for PR 10037 at commit 00ce43a.

Test build #47058 has finished for PR 10037 at commit 00ce43a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47058/
Test PASSed.

Merged build finished. Test PASSed.

@yanboliang, I am ok that we leave alias of is.na to  SPARK-12071. But it would be better you can add alias is.nan.

Test build #47065 has started for PR 10037 at commit 041c9c6.

Test build #47065 has finished for PR 10037 at commit 041c9c6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47065/
Test PASSed.

Merged build finished. Test PASSed.

Test build #47123 has started for PR 10037 at commit 95fdd2c.

Test build #47123 has finished for PR 10037 at commit 95fdd2c.

This patch fails R style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47123/
Test FAILed.

Merged build finished. Test FAILed.

Test build #47127 has started for PR 10037 at commit 3ee7d5c.

LGTM

Test build #47127 has finished for PR 10037 at commit 3ee7d5c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47127/
Test PASSed.

Thanks @sun-rui @felixcheung for the clarifications and @yanboliang for the change. LGTM. Merging this to master, branch-1.6

Thanks for your help @sun-rui @felixcheung @shivaram .


Add non regression tests

Using examples documents used in https://developers.mindee.com/docs, add non regression test through integration tests.
An action will have to be launch every day, that will only do this test on the main branch.
Each test will have to validate the summary for every implemented OTS.

[ ] Receipt V4
[ ] Invoice V4
[ ] Passport V1
[ ] Financial doc V1


[MM-TE] Fix Upgrade to Chart 5

This fixes the upgrade guide for 5.0.0 and some documentation
Fixes #234
Fixes #231

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!
/cc @jasonblais @jfrerich @emilyacook


[Question] Replace Russian localization application file (Localization_ru)

Russian localization application is no full, I am attaching a file with full localization created by the fan community
Localization_ru.txt

Great news. It's aleays nice to see how the community grows (I've already notice an increase in plays from Russian players on the Valkyrie statistics).

@estarriol

To be added in 2.5.8 this week. #1584


nodejs (devfile) and nodejs/12 (s2i) components are not accessible after push, but java-quarkus (devfile) works fine

/kind bug

What versions of software are you using?
Operating System: Linux
Output of odo version: 2.0.1
How did you run odo exactly?
Local single node OpenShift cluster
./crc status
CRC VM:          Running
OpenShift:       Running (v4.6.1)
Disk Usage:      21.31GB of 32.72GB (Inside the CRC VM)
Cache Usage:     41.01GB
Cache Directory: /home/denis/.crc/cache

"/home/denis/p/vscode-openshift-tools/out/tools/linux/odo" create nodejs n4  --context /home/denis/p/review-workspace/nodejs-devfile4 --app app1 --project sprint-review --starter
Devfile path:
Validation
 •  Checking devfile existence  ...

 ✓  Checking devfile existence [15062ns]
 •  Creating a devfile component from registry: DefaultDevfileRegistry  ...

 ✓  Creating a devfile component from registry: DefaultDevfileRegistry [13128ns]
 •  Validating devfile component  ...

 ✓  Validating devfile component [68157ns]

Starter Project
 •  Downloading starter project nodejs-starter from https://github.com/odo-devfiles/nodejs-ex.git  ...

 ✓  Downloading starter project nodejs-starter from https://github.com/odo-devfiles/nodejs-ex.git [785ms]

Please use `odo push` command to create the component with source deployed

$ "/home/denis/p/vscode-openshift-tools/out/tools/linux/odo" push --debug

Validation
 ✓  Validating the devfile [45874ns]

Creating Kubernetes resources for component n4
 ✓  Waiting for component to start [6s]

Applying URL changes
 ✓  URL http-3000: http://http-3000-n4-sprint-review.apps-crc.testing/ created

Syncing to component n4
 ✓  Checking files for pushing [695353ns]
 ✓  Syncing files to the component [753ms]

Executing devfile commands for component n4
 ✓  Executing install command "npm install" [3s]
 ✓  Executing debug command "npm run debug" [1s]

Pushing devfile component n4
 ✓  Changes successfully pushed to component

$ "/home/denis/p/vscode-openshift-tools/out/tools/linux/odo" log
time="2020-11-20T07:37:19Z" level=info msg="create process:devrun"
time="2020-11-20T07:37:19Z" level=info msg="create process:debugrun"
time="2020-11-20T07:37:24Z" level=debug msg="no auth required"
time="2020-11-20T07:37:24Z" level=debug msg="succeed to find process:debugrun"
time="2020-11-20T07:37:24Z" level=info msg="try to start program" program=debugrun
time="2020-11-20T07:37:24Z" level=info msg="success to start program" program=debugrun
ODO_COMMAND_DEBUG is npm run debug
Changing directory to /project
Executing command cd /project && npm run debug

><EMAIL_ADDRESS>debug /project
> node --inspect-brk=${DEBUG_PORT} server.js

Debugger listening on ws://<IP_ADDRESS>:5858/fad0a98a-3347-43a1-a91c-223b858a1ec5
For help, see: https://nodejs.org/en/docs/inspector
time="2020-11-20T07:37:25Z" level=debug msg="wait program exit" program=debugrun

Actual behavior
Cannot reach the app in browser

Expected behavior
Should be reachable
Any logs, error output, etc?

verified, hitting the same problem
/priority high

fixed.
Problem was that the example (starter project) was using --inspect-brk for debugging. This makes the user code break before startup (to allow debugging startup sequence). This should not be a default option.
/close

@kadel nodejs/12 s2i is deployed fine, but nodejs devfile component is still not accessible after creating it using starter project and pushing it to the cluster.


Made possible to use N Facebook Pixels Ids

The Facebook Pixel can be used with N Ids, just need to initiate them and when the event is trigged don't need to send to each ID.

Is this still something that is relevant to the project? If not this PR will be closed.

This PR make possible to use multiple Facebook pixels on the same implementation of the vue-multianalytics


no sound within browser

After published onto website,
if I play a flv media from http server, everything works as expected.
but if I play a rtsp media, there is just video, no audio,
is there anyone encountered the same issue?

What type of audio is in the stream? Locomote only supports AAC and PCMA at the moment. Could you post the SDP file?

Hi noseglid,
Thank you very much for the response.
The problem solved, it is caused by the address format of my rtsp server, since it is a combination of two video and audio address with "&&". This combined address is passed to flash player by js, so flash parsed this address as two seperate variable in loaderInfo.parameters.  So I use two seperate parameters to transport the media source and combine them later within flash, and the problem's gone.
So at last it turns out to be a pure as3 issue, not from code of locomote player itself.
Wish this experience could help other ones.
Good day!


Add a platform interface to get and set the user profile

Which is the name, locale, email, phone number, i.e. info that would
normally be stored in the users database in almond-cloud and subject to
additional verification.

Pull Request Test Coverage Report for Build 635

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 77.866%

Totals

Change from base Build 634:
0.0%

Covered Lines:
1402

Relevant Lines:
1727

💛  - Coveralls


Added Dropdown to select number of questions

Added a dropdown to select number of questions.
🛠 Fixes #11

👨💻 What's being changed:
Added a new drop down list in order to select number of questions
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

✔ Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers

📷 Screenshots

Original
Updated

Hi @Dharmik48  I forgot add a event listener it's working now..


Generic DDIC searchhelp generator is not working

Hi,
since the latest changes the generic DDIC searchhelp generator "Z2UI5_CL_TOOL_APP_SHLP_GEN" isn't working anymore.
Example: Z2UI5_CL_TOOL_APP_09
The dialog doesn't open at all.

Could this be linked to the new namespace approach?
Thanks a lot in advance.
Kind regards
Axel

fixed now with https://github.com/abap2UI5/abap2UI5/pull/635

Hi @oblomov-dev,
Great, it works again!
Thanks again for the genius support :-)
Issue can be closed.


Azure AD for non-Azure Linux boxes

Is it possible to authenticate using Azure AD on Linux boxes outside of Azure?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb8e2623-1f79-aee3-bd97-8b5f647b6980
Version Independent ID: 6346b54d-cc8d-0b79-a6b8-f9144cfa2459
Content: Log in to a Linux VM with Azure Active Directory credentials
Content Source: articles/virtual-machines/linux/login-using-aad.md
Service: virtual-machines-linux
GitHub Login: @cynthn
Microsoft Alias: cynthn

@lyda, Thanks for raising the issue. We will investigate and update you.

It is possible to use Linux devices on Azure AD if you enable Azure AD DS - Azure Active Directory Domain Services using the Azure portal.
For example: Join an Ubuntu virtual machine in Azure to a managed domain.
Azure AD Domain Services provides a managed AD domain in an Azure virtual network. You can join machines to this managed domain using traditional domain-join mechanisms. Windows client (Windows 7, Windows 10) and Windows Server machines can be joined to the managed domain. Additionally, you can also join Linux and Mac OS machines to the managed domain. When you join a machine to an AD domain, users can sign in to the machine using their corporate credentials. These machines can be managed using Group Policy, thus enforcing compliance with your organization's security policies.
On a domain-joined machine, user authentication happens using NTLM or Kerberos authentication protocols. The domain-joined machine needs line of sight to the domain controllers of the managed domain in order for user authentication to work. Therefore, the domain joined machine needs to be on the same virtual network as the managed domain. Alternately, the domain joined machine needs to be connected to the managed domain over a peered virtual network or over a site-to-site VPN/ExpressRoute connection. Thus, this mechanism isn't a great fit for devices that are mobile or connect to resources from outside the corporate network.
NOTE:
Technically, it is possible to join an on-premises client workstation to the managed domain over a site-to-site VPN or ExpressRoute connection. However, for end-user devices we strongly recommend you use either register the device with Azure AD (personal devices) or join the device to Azure AD (corporate devices). This mechanism works better over the internet and enables end users to work from anywhere. Azure AD Domain Services is great for Windows or Linux Server virtual machines deployed in your Azure virtual networks, on which your applications are deployed.


Proposal to treat burns similar to claims

Right now rune burns are behind an etching flag. Is there a reason we couldn't do burns the same way as claims (via a bit mask of the rune id in the edict?). Conceptually it feels strange they are done differently.
So for example I could have an input with 100 of rune A. First edict could do a transfer of 50 and second edict could do a burn of 50 (output field is ignored).

The etching flag turns the transaction into a burn transaction, which burns all inputs. You can easily actually burn runes more granularly by just using an edict to assign them to the OP_RETURN output.


ci: Only upload coverage when running coverage workflow, not build.

I had a pull request (#557) that changed a single line in a single integration test.  c-i reported that coverage changed:
5.72% (-48.53%) compared to a34ebfa
Where a34ebfa was the previous commit.
That doesn't sound right.  I think what is happening is that we have multiple uploads of the coverage workflow and they're fighting.
There isn't any reason to upload both anyway.

What type of PR is this?

Which issue does this PR fix:
What does this PR do / Why do we need it:
If an issue # is not available please add repro steps and logs showing the issue:
Testing done on this change:

Automation added to e2e:

Will this break upgrades or downgrades?
Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

the code coverage run showing the big change is https://app.codecov.io/gh/project-stacker/stacker/pull/557 .

Close this because https://github.com/project-stacker/stacker/pull/559


Return the receipt for restored purchases

This PR allows getting the receipt for restored purchases. This is necessary if you want to verify a restored purchase on a server.

Thanks!


Release azure-cli-acs 2.0.3

Reviewers, please verify the following for this PR:

[ ] The PR title has format Release <component-name> <version>.
[ ] setup.py has been modified with the same version as in the PR title.
[ ] HISTORY.rst has been modified with appropriate release notes.
[ ] If releasing azure-cli or azure-cli-core, __version__ defined in __init__.py should also be modified to match.

Started releasing azure-cli-acs 2.0.3...

Release of 'azure-cli-acs' with version '2.0.3' successful. View at https://pypi.python.org/pypi/azure-cli-acs/2.0.3.

GitHub release created. https://github.com/Azure/azure-cli/releases.


AnalogRX's blockade radius

In the documentation on the "digital-analog example" section, one can find the following code:
from torch import pi
from qadence import Register, AnalogRX, sample

# Global analog RX block.
block = AnalogRX(pi)

# Almost non-interacting qubits as too far apart.
register = Register.from_coordinates([(0,0), (0,15)])  # Dimensionless.
samples = sample(register, block)

# Interacting qubits as close together.
register = Register.from_coordinates([(0,0), (0,5)])
samples = sample(register, AnalogRX(pi))

We can't see any explanation why 5 should make atoms close and 15 too far apart to interact. These values are arbitrary and in the implementation of _analog_rot, the default blockade radius corresponds to omega defaulted to π. This raises the following points:

Why is omega defaulted to π?
Regardless of what it's defaulted to, shouldn't the user have access to the corresponding blockade radius to know what is close and what is too far? They could for example do something like:Register.from_coordinates([(0,0), (0, qadence.default_blockade / 2)]) # close
Register.from_coordiantes([(0,0), (0, qadence.default_blockade * 2)]) # far

This would allow them to make sure they get a specific behavior. It's true they could use AnalogRot instead, but they would loose the syntactic sugar of doing X/Y/Z rotations. Besides, functions like AnalogRX which behavior is unpredictable may be pointless.

Hey @nmheim would you be able to take that one on whenever possible ?


Migrate repository

The maintenance is not easy with many repositories.
It makes sense to gather :

OpenAPI-client-gen
openapi-server-gen
openapi-common-lib

in the repository openapi-suite.
No problem for swagger-converter-cli and swagger-validator-cli we don't need often to update these packages.

done


ReSharper unit tests detects errors that VS2022 don't.

One that is in the bellow image is related with DefaultThreadCurrentUICulture. Naturally VS2022 enables "en-US" by default for unit tests but it shouldn't. Imagine we want to rune all tests based on a certain culture we need to set it explicitly and this is the right way, in my opinion.

As @tig already spoke before, another issue that happens a lot on VS2022 is silently, on the failed unit tests or threads not stopped and closed, by maintained the current dotnet session opened and causing malfunction forcing us to restart VS2022. ReSharper Unit Test detect it and popup a message box (see bellow) asking if the dotnet process is to be kill or wait. Thus this help us to the need to discover what is causing the dotnet process no finishing.


Make configurable default bblfsh UAST flavour

To mitigate this problem: https://github.com/src-d/gitbase-spark-connector/issues/76
we should be able to configure the UAST default output type from an environment variable.

Preferably not: environment variables might be controlled by a different user (e.g. in our case, infra, not developer/analyst), then some queries will be not valid depending on such deployment config. This is the same problem we had for regexps.

we will limit the usage of UAST UDF at GSC level.

we will limit the usage of UAST UDF at GSC level.


pip install dependencies missing

@rragona tried to install tofu on a Macbook using a fresh (virgin) conda virtual environment

The conda package for Mac doesn't exist so he had to use a pip install (using the pip installed via conda)
pip did not check the tofu dependencies and threw errors at every try

=> It appears we forgot to include a requirements.txt file that pip uses to check the dependencies
=> see this site  for details on pip dependencies managing
To try and reproduce : create an empty ne conda package on a mac (with python 3.7, including just the basics like conda, pip).
Then try installing tofu using pip install tofu
=> it should complain about missing dependencies (due to missing requirements.txt)
=>and Ricardo also has an issue because he doesn't have gcc-8 (is it the only solution ? he has another compiler called LLVM)
Also, it seems we forgot openmp in the dependencies for installation

Thanks @Didou09 for your help and for opening this issue.
This is the error I have while trying:
pip install tofu

Collecting tofu
  Using cached https://files.pythonhosted.org/packages/5a/06/cbb85978151b0854bd2d2f022cfad1ff15a77db88593aeb236cf671bcd89/tofu-1.4.1.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /Users/rik/anaconda/envs/tofu/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/51/1mbjm0lj6098n3_rv0g7tfdr0000gn/T/pip-install-lw_tfc9_/tofu/setup.py'"'"'; __file__='"'"'/private/var/folders/51/1mbjm0lj6098n3_rv0g7tfdr0000gn/T/pip-install-lw_tfc9_/tofu/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/51/1mbjm0lj6098n3_rv0g7tfdr0000gn/T/pip-install-lw_tfc9_/tofu/pip-egg-info
         cwd: /private/var/folders/51/1mbjm0lj6098n3_rv0g7tfdr0000gn/T/pip-install-lw_tfc9_/tofu/
    Complete output (18 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/51/1mbjm0lj6098n3_rv0g7tfdr0000gn/T/pip-install-lw_tfc9_/tofu/setup.py", line 151, in <module>
        openmp_installed = not check_for_openmp(os.environ['CC'])
      File "/private/var/folders/51/1mbjm0lj6098n3_rv0g7tfdr0000gn/T/pip-install-lw_tfc9_/tofu/setup.py", line 139, in check_for_openmp
        stdout=fnull, stderr=fnull)
      File "/Users/rik/anaconda/envs/tofu/lib/python3.7/subprocess.py", line 339, in call
        with Popen(*popenargs, **kwargs) as p:
      File "/Users/rik/anaconda/envs/tofu/lib/python3.7/subprocess.py", line 800, in __init__
        restore_signals, start_new_session)
      File "/Users/rik/anaconda/envs/tofu/lib/python3.7/subprocess.py", line 1551, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'gcc-8': 'gcc-8'
    cython version = 0.29.13
    numpy  version = 1.17.3
    cython version = /Users/rik/anaconda/envs/tofu/lib/python3.7/site-packages/Cython/__init__.py
    numpy  version = /Users/rik/anaconda/envs/tofu/lib/python3.7/site-packages/numpy/__init__.py
    ................ checking if openmp installed...
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Hey @rragona
I added a requirements.txt for the release that's coming in the next few days.
Also the error regarding gcc-8 should be solved.
But if you want we can chat a bit on how to make it work in the meantime: https://gitter.im/ToFuProject/community

Hi guys,
Just a thought, but as far as I understand the machinery, requirements.txt is not necessary.
Why? Because setup.py already includes dependencies.
As far as I can tell, the error is due to the heuristic that checks for the existence of the compiler and not the dependencies.
My guess is that this will not solve any problems but adding the file will confuse us all...
Sorry for the strong opinion, but I really believe this has nothing to do with requirements.

Okay, so after reading this again, I have to take some of what I said back. What is the current status of Mac OS as a platform? Are we not recommending to install from source in the docs? And not from pypi?

As far as I can tell, the error is due to the heuristic that checks for the existence of the compiler and not to the dependencies.

I talked directly with Didier, apparently they first tried a pip install that failed because it couldn't find the requirements.txt file. They installed this by hand and then got the error mentioned above, which does come from an error from my part in the setup.py that is now corrected in the devel branch. In short: there were 2 problems when installing with pip on Mac, the error message has nothing to do with the requirements file

Are we not recommending to install from source in the docs?

Yes, I also mentioned this to @Didou09. But it seems like installing directly from pip should also be a possibility.
I should also add the requirements.txt is quite standard and updates in github the dependencies. So I see no problem in adding it.

I'm still thinking something about this is weird. In some of my other projects, I've defined dependencies in the setup.py and made wheels that registered the right dependencies. There was no need to add requirements.txt. I'm puzzled about why it is necessary here. Do you have any explanation why it works with it?
The disadvantage of adding this file I see is that there are now two places where requirements are defined. Hence when changing these, we will run into the risk of forgetting to update. It adds easy to miss complexity. Couldn't it be that it's a false positive?
Final thing I don't understand: pip install did not work at first but by adding local files it started to work? This is a weird way of installing: neither pure pip nor pure from source. For me it looks hacky / I'm thinking I'm missing something.

So, for the record, I have symptoms when installing from scratch under MacOS, linked to the compiler for the Cython parts.
Following https://tofuproject.github.io/tofu/installation.html#installing-as-a-developer, I have noticed two things:

pip install -e .[dev] does not work in my shell by pip install -e ".[dev]" does
it hangs due to gcc throwing an error, which was first resolved by running xcode-select --install but still fails later due to this:

tofu/geom/_vignetting_tools.cpp:610:10: fatal error: 'ios' file not found
#include "ios"
So as of right now, I'm unable to install tofu on my Mac...

So:

What was done: empty conda environment, installing from source (pip). It failed because pip could not find the requirements.txt and dependencies were missing. So dependencies were installed by hand (conda install) and finally pip was retried, it failed this time because of the error in the setup.py, now fixed, that Laura mentioned.

What you are saying @flothesof is that pip does not need the requirements.txt when you create a pip wheel. But this was not a pip wheel. It was installing from source using pip. We never created pip wheels. Maybe we should though ? (separate issue in that case)

I agree storing dependencies in separate places is a source of complexity. But it is already the case. The dependencies are in the setup and also in the conda recipe (for creating the conda package).

I feel that, by principle, installing from source with pip should always be possible and easy. What do you think ?

=> So, although it does introduce a bit of extra complexity, it does seem useful to add the requirements.txt as Laura did for non-developpers who wnat to install on Mac or windows from source using pip.
Complexity is an issue, but tofu being a difficult to install for newbies is a higher-priority issue. If requirements.txt make it easier for them I feel it is the way to go.
Furthermore, the requirements.txt being very short and simple could be automatically generated through Continuous Integration in a near future, what do you think ?

Status report after discussing this with @Didou09:

The goal is to get pip install tofu to work on Mac
there is a distinction between build requirements (Cython, numpy which are imported by setup.py and therefore necessary to install from source) and user requirements (others already present in setup.py, later user in tofu itself like matplotlib, scipy)
adding requirements.txt seems to solve the build requirements issue (on @lasofivec 's machine) but it's not clear why
a better hotfix would be to upload a wheel to PyPI obtained using python setup.py bdist_wheel

As a side note, I was able to compile from source on Mac without requirements.txt, which I interpret to show that we don't need it...

Thanks for the status report,
so we go for both (requirements.txt + bdist wheel) for the next release to try and make this as robust as possible.
We'll see later whether we can afford to remove the requirements.txt to reduce the complexity

I've just tested the wheel build. On my machine, I am able to:

create a .whl file in a Py36 env
create a new test conda environment without any dependencies except pip (also Py36) and pip install path/to/.whl
I am then able to run python and import tofu

Another test would be that I send the .whl I've built to @lasofivec to make sure this also works.

Yes, building a wheel is a good complemnetary to adding the requirements.txt (ceinture + bretelles). If it works, we can upload it to Pypi.
Could you by any chance try to build a conda package for Mac ? (low priority, but just in case)

Also, I saw that the OSX platform seems to have been added recently to the Travis CI available platforms.
@lasofivec , do you think you could have a look at this link and see if we can create a build Matrix in the .travis.yml that would also build on Mac OS ?

Ok I'm sorry I'm arriving a bit late to the conversation...

Final thing I don't understand: pip install did not work at first but by adding local files it started to work?

It wasn't adding files but installing the "requirements" (dependencies) "by hand". As Didier explained.

Following https://tofuproject.github.io/tofu/installation.html#installing-as-a-developer, I have noticed two things:

I think you (@flothesof) re-wrote the doc for Mac. When the doc was in tabs form, it was written by me and it wasn't at all the same procedure. As you are more familiar with packagings, I left it unchanged. We will have to update the doc in the next release whichever way we decide it's the best on Mac.
Installing the Xcode utilities was in the original version of the doc (I think).
I don't get the same bug, if you want we can see what's different between our configurations, but I would open a different issue for this (I'm pretty sure is related to Mojave).

Status report after discussing this with @Didou09:

The goal is to get pip install tofu to work on Mac
there is a distinction between build requirements (Cython, numpy which are imported by setup.py and therefore necessary to install from source) and user requirements (others already present in setup.py, later user in tofu itself like matplotlib, scipy)

Yes, I added scipy and matplotlib since they are quickly needed for users and they match the requirements in the setup.py
I am not sure if you mentioned the difference so that we take the libraries only needed when using tofu out of the requirements.txt, or if you want me to write them in a separate category on the file, or just as a "FYI" ?

adding requirements.txt seems to solve the build requirements issue (on @lasofivec 's machine) but it's not clear why

I dont understand why is there a confusion ? "pip install tofu" without the requirements file doesn't work because it says there are dependencies missing, when you tell pip which dependencies are missing (with the use of requirements.txt), it install them, then installs tofu...
I think you should also be able to try it with the version with and without the file.

a better hotfix would be to upload a wheel to PyPI obtained using python setup.py bdist_wheel

As a side note, I was able to compile from source on Mac (CFLAGS='-stdlib=libc++' pip install -e ".[dev]") without requirements.txt, which I interpret to show that we don't need it...

"from source" but also with an environment from scratch ? tofu was always installable if you already had the right dependencies in you environment/machine.
ok for the wheel :)

I've just tested the wheel build. On my machine, I am able to:

create a .whl file in a Py36 env
create a new test conda environment without any dependencies except pip (also Py36) and pip install path/to/.whl
I am then able to run python and import tofu

Another test would be that I send the .whl I've built to @lasofivec to make sure this also works.

As we talked by email, I obtained an error during installation that didn't prevent the installation of tofu from happening but that for me remains unexplained.
ERROR: pytools 2018.5.2 requires decorator>=3.2.0, which is not installed.
For the moment I would suggest keeping both suggestions, but I trust you judgment. If you want to get rid of the requirements.txt and believe the wheel is robust enough, I agree! :)

Also, I saw that the OSX platform seems to have been added recently to the Travis CI available platforms.
@lasofivec , do you think you could have a look at this link and see if we can create a build Matrix in the .travis.yml that would also build on Mac OS ?

this is already an issue somewhere else and, in my opinion, separated from this one :)
but yes

The requirements.txt was added, in devel, by PR #251
It is a hotfix, but the outcome of this discussion is that we should probably discuss all these aspects (CI / installation / deployments on various platforms) in more details in a near future. And open a dedicated issue or even project / milestone.

Adding to this conversation just FTR, I recalled why the compiler used with OS X was set manually:
https://github.com/ToFuProject/tofu/issues/183
So @rragona It will be good if you are sure to use the latest gcc version available for your distribution.

Just a small update on this topic.
I have set up a clean conda environment on OS X:
conda create --name tofu python=3.7 ipython -y
source activate tofu
I have downloaded requirement.txt from the devel branch and executed:
pip install -r requirements.txt tofu
I have now a working tofu!
Python 3.7.5 (default, Oct 25 2019, 10:52:18)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.9.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import tofu as tf
~/anaconda/envs/tofu/lib/python3.7/site-packages/tofu/imas2tofu/__init__.py:27: UserWarning:

IMAS python API issue
imas could not be imported into tofu ('import imas' failed):
  - it may not be installed (optional dependency)
  - or you not have loaded the good working environment

    => the optional sub-package tofu.imas2tofu is not usable

  warnings.warn(msg)
~/anaconda/envs/tofu/lib/python3.7/site-packages/tofu/mag/__init__.py:28: UserWarning: Traceback (most recent call last):
  File "~/anaconda/envs/tofu/lib/python3.7/site-packages/tofu/mag/__init__.py", line 11, in <module>
    import imas
ModuleNotFoundError: No module named 'imas'

    => the optional sub-package tofu.mag is not usable

  warnings.warn(msg)

In [2]:

I have also repeated in an another clean environment and this procedure works smoothly.
CAVEAT: Following the suggestion of @lasofivec I have installed gcc-8 from homebrew.

Awesome :) thanks for the report ! we are currently working with Didier on how to make it easier !


Document page 404

Describe the bug
404 on page https://adminbro.com/DropDown.html

thanks - it was posted on our slack channel as well: #406 - try https://adminbro.com/Dropdown.html until I fix it


Allow users to create database assetstore 'files'

Allow users to create database files.
Currently, to create a file an admin must first create an assetstore which references a specific database.  The assetstore import function is used to create a Girder file that references a specific table/collection in the database with a default filter, fields, sort, limit, and format.
In addition to this, anywhere uploads are allowed we could alternately allow adding a database file.
There are a number of features that would be desired to implement this.  One a basic implementation is available, some of these can become their own issues.

[x] For the database assetstore plugin, add a settings page.  This would

[x] have a checkbox to enable user file creation
[x] optionally allow any user with write privilege to add a file
[x] have a list of acls which specify which users are allowed to create files
[x] for any given acl (and the general case), allow a regex to validate the database uri.  This can be used to limit the hosts, database types, etc.

[x] Internally add a special assetstore ID that is used for user-defined files.  This should be a constant so that if the plugin is disabled and reenabled no changes need to be made.
[x] Add button on upload dialogs or where upload buttons occur to switch to the create database file dialog
[x] Add create database file dialog.  This is a similar to the current assetstore import functionality, but needs to ask for a database uri (and does not need to have the destination fields, since the upload button already has this as a context), and would only allow one table/collection at a time.
[ ] Add a button to check access to the database uri (both in the file dialog and the create assetstore panel).  This verifies that the database is reachable at the time the user is adding the file (it can be optional)
[ ] Add a query builder to help construct fields, sort, filters.  This would not support general functions at this time.  See the query builder that was added to Minerva for an example of how to do this.
[ ] Show the current default limit (try to report what it might be based on the current data with the current filters)

There is a WIP branch that allows a user on the web client to edit files after they have been uploaded.