support for GA2M models/FAST/pairwise interactions (mltk)

Hi wonder if supporting ga2m/fast is on the roadmap? thx!
Accurate Intelligible Models with Pairwise Interactions
www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf
https://github.com/yinlou/mltk/wiki/Generalized-Additive-Models#ga2m
https://www.datarobot.com/blog/interpretability-and-accuracy-or-how-to-have-your-cake-and-eat-it-too/

@rquintino i was not aware of ga2m/fast algorithms. i'll give them a read.
The roadmap certainly includes GAMs with feature interactions via tensor products of splines.
I wonder if this is similar to ga2m?

I would also definitely be interested in implementation of feature interactions

Here's a link to ga2m model:vhttps://dl.acm.org/citation.cfm?id=2487579
Please please consider similar functions like in mgcv, such as s(x, by = z) in addition to the tensor products of spline.

@rquintino @bglick13 @syhuang-veolia
im sorry to say that for now GA2M and FAST are not on the feature roadmap.
the main reason is that these appear to require decision trees as the base learners, not spline functions.
however feature interactions are (slowly) in development!

@rquintino @bglick13 feature interactions are here! (pygam >= v0.6.0)
you can use tensor products to fit interactions using the te() function like so:
GAM(te(0,1)).fit(X, y)

which fits a tensor product of splines on features 0 and 1.
you can also get more specific like so:
GAM(te(s(0, n_splines=100), f(1))).fit(X, y)

which fits a tensor product between a spline-term with 100 basis functions for feature 0, and a factor term for feature 1.
@syhuang-veolia i've added by-variables so that you can make models like:
GAM(s(0, by=1)).fit(X, y)

which fits a spline on feature 0, using feature 1 as a by-variable


cannot start threebot builder

error while trying to start threebot builder

Branch: development
Commit_ID:  23d753f738abbc7add99b98b6c23619ab20671d2

can't be the case anymore.


Update Editor State Without De-initialising

This is definitely possible, I had a tab open about the process... but I have either closed it or it was on my mobile and I'll... maybe I'll find it in history.
It's a relatively new method that was added to editor whereby the content can be replaced on the fly.
Doing this should eliminate the stutter currently present when switching between documents.

Simply removing the nullification of article before nav (22965ca15088ebc9ca365c097feaff329df02d5a) removes the stutter. This was originally added to ensure that the editor state changed, but a :key was added to the form to ensure that instead (which works, provided the article is updated beforeRouteUpdate - and not just in the mounted step).
What it introduced, however, are errors received from Tiptap editor about being unable to append a node...
...which... is perhaps indicative of the fact that article and therefore the document is being updated before the call to deinit and re-initialise the editor.
I experimented with an update Document call a little on Tiptap but this... triggered my save functionality and overwrote the current article. So... that was based on a watch hook on the document itself, so either... that needs fixing or, perhaps by checking against ID as well as doc state... or... something else, like, maybe the way this is set up needs a more dramatic rewrite.

Simply removing the nullification of article before nav (22965ca15088ebc9ca365c097feaff329df02d5a) removes the stutter. This was originally added to ensure that the editor state changed, but a :key was added to the form to ensure that instead (which works, provided the article is updated beforeRouteUpdate - and not just in the mounted step).
What it introduced, however, are errors received from Tiptap editor about being unable to append a node...
...which... is perhaps indicative of the fact that article and therefore the document is being updated before the call to deinit and re-initialise the editor.
I experimented with an update Document call a little on Tiptap but this... triggered my save functionality and overwrote the current article. So... that was based on a watch hook on the document itself, so either... that needs fixing or, perhaps by checking against ID as well as doc state... or... something else, like, maybe the way this is set up needs a more dramatic rewrite.

fixed by 25bc82b071526c145c1c5f7944ef0f771a3818fd

fixed by 25bc82b071526c145c1c5f7944ef0f771a3818fd


randomize initial zone selection

What is the problem that this PR is trying to fix?
Workers, when deployed, are bound to a primary zone. Whenever a job (and VM) is started, the primary zone is used first, then an alternate zone is selected.
The core problem is when we start hitting zone exhaustion errors for a zone. Since there are pools of workers per zone, and each worker will try its primary zone first, which just puts more pressure on the api, raising the risk of api rate limit issues.
What approach did you choose and why?
This is a first step towards a fix, but not yet the full fix.
This changes the concept of a primary zone, and instead has a random zone picked each try. A zone can still be defined in the config, but this zone will be used each and every time.
The follow up work to this PR is to block a zone for 10mins across all workers (possibly using Redis), which will help reduce api usage, and a better self healing worker setup.
How can you test this?
I've tested this locally by connecting it to staging, and it worked a treat.
What feedback would you like, if any?
This is for general discussion
I also need help understanding how to fix the test failures

The core problem is when we start hitting zone exhaustion errors for a zone.

Which zone exhaustion errors are you referring to? IIUC, most resource quotas (as well as API quotas) are per project and/or per region rather than per zone. (See GCE Quotas page)
Example from recent outage:
Error
QUOTA_EXCEEDED: Quota 'SSD_TOTAL_GB' exceeded. Limit: 800000.0 in region us-central1.

Since there are pools of workers per zone, and each worker will try its primary zone first, which just puts more pressure on the api, raising the risk of api rate limit issues.

I think this is only the case when a zone is specified in the worker config. In our case it looks like instances are pretty well distributed across zones already, no?

I wonder if it would make more sense to have each worker create job instances in its own zone, if possible, since those are already automatically distributed by the managed instance group.
Also, not all resource types (in particular GPUs) are available in all zones. So I'm not sure it makes sense to make zone pinning an all-or-nothing thing, because I believe that will cause problems when GPUs (and perhaps some specific CPUs) are specified.

Edit: I take it all back, I see there were plenty of these errors during the last outage:
Mar 29 23:32:53 production-2-worker-org-gce-4mr9 travis-worker-wrapper:
time="2019-03-30T04:32:53Z" level=error msg="couldn't start instance, attempting
requeue" err="code=ZONE_RESOURCE_POOL_EXHAUSTED location=
message=The zone 'projects/travis-ci-prod-2/zones/us-central1-c' does not have
enough resources available to fulfill the request.  Try a different zone, or try again later."
job_id=123456798 job_path=xyz/xyz/jobs/123456798 pid=1
<EMAIL_ADDRESS>repository=xyz/xyz self=step_start_instance start_timeout=8m0s uuid=84c6f99e-f021-4b13-baf1-ba101c22e3ab

Thanks for the feedback AJ.
I'm just about to hit the hay, but I thought I would also add that our
Google reps recommended temporarily not using a zone when we hit these
errors.
I'll write up a more detailed reply when I wake up.
On Mon, Apr 1, 2019 at 11:50 PM, AJ Bowen<EMAIL_ADDRESS>wrote:

Edit: I take it all back, I see there were plenty of these errors during
the last outage:
Mar 29 23:32:53 production-2-worker-org-gce-4mr9 travis-worker-wrapper:
time="2019-03-30T04:32:53Z" level=error msg="couldn't start instance, attempting
requeue" err="code=ZONE_RESOURCE_POOL_EXHAUSTED location=
message=The zone 'projects/travis-ci-prod-2/zones/us-central1-c' does not have
enough resources available to fulfill the request.  Try a different zone, or try again later."
job_id=123456798 job_path=xyz/xyz/jobs/123456798 pid=1
<EMAIL_ADDRESS>repository=xyz/xyz self=step_start_instance start_timeout=8m0s uuid=84c6f99e-f021-4b13-baf1-ba101c22e3ab
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/travis-ci/worker/pull/583#issuecomment-478530902, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAh_QtAMyI8vI5Goe_0HV_clixYcSVkks5vceRtgaJpZM4cVTDf
.

Thanks for the feedback AJ. I'm just about to hit the hay, but I thought I would also add that our Google reps recommended temporarily not using a zone when we hit these errors. I'll write up a more detailed reply when I wake up.

This definitely makes more sense now that I know we had hit ZONE_RESOURCE_POOL_EXHAUSTED errors during the last incident. :+1:


chore: Update to be triggered when tag pushed

Description

Update to be triggered when tag pushed

Checklist

[x] My code follows the existing style, code structure, and naming taxonomy
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have performed a self-review of my own code and included any verifying manual calculations
[x] I have added or updated unit tests that prove my fix is effective or that my feature works, and achieves sufficient code coverage.  New and existing unit tests pass locally and in the build (below) with my changes
[x] My changes generate no new warnings and running code analysis does not produce any issues
[x] I have added or run the performance tests that depict optimal execution times
[x] I have made corresponding changes to the documentation

On a related note, I’ve modified the repo tag protection rules to basically prevent accidentally over-incrementing to future major version numbers, set as [2-9]+.*

Look like tag 1.1.0 auto-removed itself eventually after you delete the release note, unless you did something separately.  I didn’t.

@DaveSkender

On a related note, I’ve modified the repo tag protection rules to basically prevent accidentally over-incrementing to future major version numbers, set as [2-9]+.*

Thanks! And I removed the tag yesterday


UI TODO: Duration field in event creation should not be in hh:mm format

@SELord Did the client want this to be a dropdown?


Fix links in tutorial

In this tutorial, two links are not rendered correctly.

Converting those <a href ... tags to markdown style links would fix the issue.

Hi @Yash-10

I think the changes have been made already.

Please share links to actual code instead of screenshots:
https://github.com/poliastro/poliastro/blob/eedc5f1c810b748b7b0bbc719c2147371b0abba3/docs/source/examples/Propagation using Cowells formulation.mystnb#L24
It's interesting that nested HTML got badly translated. Perhaps this is a bug in myst-parser? What do you think @DhruvSondhi ?

Hello @astrojuanlu, Yes this seems to be off ... Though if we check here, it needs to work as intended ... I will test it out a little bit before I can comment on it as a Bug 🤔 ... I will be sharing my findings here as well 😄

Yes, @astrojuanlu, You are absolutely correct ... It seems to be a bug with MyST-Parser ... The notebook is working correctly with all the nested HTML tags (can be seen here ... It seems to be some issue with the parser for the same 🤔

Thanks @DhruvSondhi for confirming. I think we should create an issue in the upstream repository or wait to investigate more?

I am trying to find the cause of the issue, so I would suggest that we wait a little bit before making the required conclusions & hence the issue. I am trying to create a case for the same in another notebook and share my findings for the same here ... After that, we can create an Issue in the Upstream Repository

Yeah, sure!!

Thanks for confirming @DhruvSondhi ! No rush, let us know when you are done with your investigation

@DhruvSondhi based on the link you have shared, I have seen the following markdown syntax for external tags being used:

which is then rendered as:

This is the same format which I have used in my follow-up PR.

Yes, it seems to be an issue ... The MyST parser is not able to parse the nested HTML Tags. Using anything else other than the HTML or native Markdown is breaking the building procedure ... I have tried a lot of combinations with {note}, {admonition} , etc but these first of all don't render correctly in the Jupyter Notebook, second of all, it doesn't' all the parser to continue the build. I also tried doing it with other nested HTML tags & it doesn't seem to work ... Another thing that can be tested is by doing something like ```html ``` but this doesn't render correctly in the notebook ... So, according to me, it is a BUG ... Though I will try it once again tomorrow with some other combinations. This is only a problem with the Cowell's notebook 🤔


setLooping(true) ???

How can I set an audio in loop?

Is there anyone to help?

Hey @mehyaz , Is it something like repeat song,
currently if you add songs to the playlist, and the playlist reaches the end of the list, by default it starts playing the list from the start, which is kind a like repeat all songs in the list.
You can take a look at JcAudioPlayer.java in line 114 the nextAudio() funtion you will see in line 126 : playAudio(playlist.get(0));
which means that if there is an exception like u reached something like the end of the list, it starts playing from the begining again.
If you want to work on this, here is a small suggestion you can start off with
Logic/Algorithm to get Working with the Repeat Funtion

We will need a String field variable "repeatMode" in the JcAudioPlayer class which can be set to NONE/ALL/ONE  by default it can be ALL. This varaible tells us whether we are Looping the Playlist or not.
Have a button on the PlayerView layout XML with appropriate icons and assign them to the respective repeat button "repeatBtn" just like the play/prev/next button. and in the Listener repeatBtn.setOnClickListener - on Click of the button Check the current State of the repeatMode and change it accordingly. LOOK AT point 4.
Now in the nextAudio() inside the JcAudioPlayer class checkout line 120: JcAudio nextJcAudio = playlist.get(currentPositionList + position);

before doing this do something like this
if(repeatMode.equals("NONE")||repeatMode.equals("ALL"))
{
JcAudio nextJcAudio = playlist.get(currentPositionList + position);
// this means as usual just play the next audio
}
else if(repeatMode.equals("ONE"))
{
JcAudio nextJcAudio = playlist.get(currentPositionList);
// this means repeat the audio
}
Now in the exception block
line 126: playAudio(playlist.get(0));
do this
if(repeatMode.equals("ALL"))
{
playAudio(playlist.get(0));
//repeat all the songs in the list
}
else
{
jcPlayerService.stop();
// don't repeat the list
}
//String currentRepeatMode="ALL"; declare this somewhere
4. You can create a public method setLooping(String mode) inside the JcAudioPlayer class and call it on the click of the repeat button
inside the handler for the repeatBtn onClickListener
if(currentRepeatMode.equals("ALL"))
{
currentRepeatMode="NONE";
setLooping(currentRepeatMode);
}
else if(currentRepeatMode.equals("NONE"))
{
currentRepeatMode="ONE";
setLooping(currentRepeatMode);
}
else
{
currentRepeatMode="ALL";
setLooping(currentRepeatMode);
}
DONE...HOPE THIS COULD GIVE EVERYONE AN IDEA ON WORKING WITH LOOPING AUDIO IN THE PLAYLIST>>
@jeancsanchez please take a look at this also..it might be helpful, Thank You.


api: v2alpha1: add total field to all List responses

Add field with total count of records for accounts, atxs, layers, rewards and transactions.

bors merge

bors merge

bors merge


Filesystem problems: Consolidated

I am creating a consolidated thread here so that file system issues can be looked at in one place, good for both users and Mods who have to look at them. WSL is a work in progress, specially on the file system side. Please comment here first before creating a new issue to verify that your problem is new. Mods, please delete if you disagree. I would vote for closing the referenced issues wherever they are duplicates (Not #216 e.g.)
Inotify

Inotify not working: #216 (Partial fix is coming to insiders, causes triggers only when changes are from inside WSL)

Inconsistent WSL folder

Files inside WSL folder when edited on Windows side will not see the changes reflected on WSL side.
File created on Linux inside WSL folder, but deleted from Windows will get into a Limbo.
ONLY KNOWN FIX AS OF NOW: Don't make changes to files inside WSL from Windows.
Related Issues: #45 #403 #276 #552 #942 #678 #659

Inconsistent Windows folder

See below. Possible false flag

Its very likely that I'm doing something really stupid here but I see that my Windows fs is inconsistent in terms of actual content. Can someone verify (or tell me what I'm doing wrong)?
Steps:
Powershell
PS C:\Users\WINDOWSUSERNAME\projects\sample\cpp> cat .\Makefile
main: main.cpp

WSL:
[X220 /mnt/c/Users/WINDOWSUSERNAME/projects/sample/cpp]cat makefile
all:
        g++ $(P).cpp -o $(P)

Expected:
Both should have same data
Seeing:
Both have different data, one has main: main.cpp, the other has g++ ...
Windows ver:
Microsoft Windows [Version 10.0.14393]

@SRGOM Yesterday I ran into a problem, seemingly random, in the management of access rights.
I have many directories mapped through symlink to the Win side and yesterday the MonoDevelop could not modify the files contained in one of them.
I haven't reported it because I can't figure out how to replicate it.
I also had the problem of the files created and/or modified on the side Win not visible from WSL but I don't opened a thread because he had already been reported.

Cannt umount ntfs drive:
Frederick @ BEAST: ~ $ mount
Mount:/proc/self/mountinfo: parse error: ignore entry at line 17.
 *** ERROR ***
umount:/proc/self/mountinfo: parse error: ignore entry at line 18.
Frederick @ BEAST: ~ $ ls-al/mnt
total 20
drwxr-xr-x 2 root root 0 Sep 5 14:12.
drwxrwxrwx 2 root root 0 Sep 2 14:34 ...
drwxrwxrwx 2 root root 0 Jun 12:12 4 c
drwxrwxrwx 2 root root 0 Jun 12:12 4 d
drwxrwxrwx 2 root root 0 Sep 5 14:12 x * * * X IS STILL HERE ***
If i try to disconnect it from Win I get this becaus X is still in use:

Just a quick note here until the Devs get back from Labor Day Weekend:
Makefile and makefile are different files to NTFS and Linux/WSL, but until lately Windows has been blocked from making both files in the same folder because Windows is insensitive to difference in case. It still gets confused because it can't tell the difference. Linux relies on this behavior (case sensitivity), so it has been made available on the WSL side. Since Windows doesn't handle it very well, when you use it in a folder  on the WSL side, don't use that folder with Windows, or if you do, expect strange things to happen.
The VolFS file system (all file paths that don't begin with /mnt, such as /home) has been set up to be much closer to Linux standards than is normally available under Windows. Consequently the Windows side hasn't been able to create files properly in VolFS, and it may never get this capability. %LOCALAPPDATA%\lxss has been hidden from Windows, which some might think is a clue not to use Windows there. If you need to access files in VolFS (e.g. /home/whatever) first use Bash.exe to copy them to someplace that begins with /mnt, then use Windows to your heart's content on them, then use Bash.exe to copy them back when you are done. (At least for now.)
Yes those are inconsistencies, but they are pretty well known. Whether they are bugs or features is another story. If you accept the two rules (don't use the WSL to create both Makefile and makefile in folders where Windows might need to read them, and don't use Windows to access files in %LOCALAPPDATA%\lxss) things actually aren't too bad. If you break those rules, inconsistencies  can be expected.
You might be able to get more traction suggesting how those design decisions should have been handled on the Uservoice site.
The handling of symlinks and hardlinks has been problemmatic. The current design seems to rely on different functionality for it between the WSL side and the Windows side. The design decision that was made says that NTFS symlinks and other reparse points probably won't work from the WSL side, and symlinks and hard links made while you are in Bash.exe (WSL) won't work on the Windows side. Some of us think that at least existing NTFS reparse points should be honored/followed when encountered in WSL (assuming permissions allow) even if you can't create or change them while in WSL. But that's not how the devs implemented it. So for now, you might want to use a naming scheme or something so you can tell which ones were made (and can only be used) in WSL, and which are reparse points in Windows that won't work properly in WSL.
Good luck enumerating these and other inconsistencies.

@rodrymbo

Consequently the Windows side hasn't been able to create files properly in VolFS, and it may never get this capability.

I've managed to get a guest share running of Samba's smbd that is plausibly accessible from Windows. That provides an interesting avenue to this overall problem.

@GSam - it's actually pretty simple to get the ssh server running, at which point one can use sftp to access files in VolFS. Several programs I know and use (e.g. Notepad++) can access SFTP servers pretty much as readily as local filesystems.
Congrats on getting Samba running! Don't say it too loud or others will ask you how you did it. ;)

Yeah, @GSam, how you did it? Using some patched version? Original smbd failed to start because can't to get network interfaces...


go build may fail on macOS and Ubuntu

What happened:
When running go build -o terraform-provider-rubrik the follow error message may show
./main.go:10:15: cannot use "github.com/rubrikinc/rubrik-provider-for-terraform/rubrikcdm".Provider (type func() "github.com/rubrikinc/rubrik-provider-for-terraform/vendor/github.com/hashicorp/terraform/terraform".ResourceProvider) as type "github.com/hashicorp/terraform/plugin".ProviderFunc in field value
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):

Environment:
macOS and Ubuntu

Issue confirmed due to GOPATH issues


Unrecognized options running make (--enable-words-int)

I follow README and got following error:
checking for charset conversion... iconv
configure: creating ./config.status
config.status: creating Makefile
config.status: WARNING:  'Makefile.in' seems to ignore the --datarootdir setting
config.status: creating config.h
config.status: config.h is unchanged
configure: WARNING: unrecognized options: --enable-words-int

and after run make:
$ make make: *** No rule to make target '../libjulius/libjulius.a', needed by 'julius'.  Stop.
is there any solution because I have compiled also the libjulius using make and succeed?

I have successfully compiled Julius now, but still annoyed with this statement
configure: WARNING: unrecognized options: --enable-words-int

This directive controls the maximum number of words in the dictionary.
Words count is a 16 bit unsigned number in default. You can use integer
instead of word by this directive. That's all.
aviezab<EMAIL_ADDRESS>ezt írta (időpont: 2019. aug. 2., P
15:35):

I have successfully compiled Julius now, but still annoyed with this
statement
configure: WARNING: unrecognized options: --enable-words-int
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/julius-speech/julius/issues/114?email_source=notifications&email_token=AHNHTWXBCWKFT4OAHIB7GOLQCQZZTA5CNFSM4II5WOX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3NYKVY#issuecomment-517702999,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHNHTWULEN6DSW422WPFGP3QCQZZTANCNFSM4II5WOXQ
.

@Estalhun I can't compile with enable words int as this error is clearly defined:

If you put #define WORDS_INT into your code, this code below will do the job for you.
Current stddefs.h:
#ifdef WORDS_INT
/* maximum number of words = 2G = 2^31 (--enable-words-int) /
typedef int WORD_ID;		///< Typedef for word ID
#define MAX_WORD_NUM<PHONE_NUMBER> ///< Maximum size of vocabulary
#define WORD_INVALID<PHONE_NUMBER> ///< Out of word ID to represent no-existence of word
#else
/ maximum number of words = 65535 */
typedef unsigned short WORD_ID; ///< Typedef for word ID
#define MAX_WORD_NUM 65535	///< Maximum size of vocabulary
#define WORD_INVALID 65535	///< Out of word ID to represent no-existence of word
#endif

@Estalhun just to clarify, sttdefs isn'st same with stddef isn't it?
I found stddefs.h at libjulius folder. Gonna try your advice, thanks, Estalhun.

Please run configure at the top directory.  The option "--enable-words-int" is valid only for libsent/configure.

@nitslp-ri Thank you for explanation.
I will try, and also let me make clear in readme.md as pull request later.


Feature/collection bulkadd

Adds a Django admin action for bulk adding selected digitized works for collections (#16), as well as associated views and tests.

Codecov Report

Merging #22 into develop will increase coverage by 0.08%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop      #22      +/-   ##
===========================================
+ Coverage    98.18%   98.27%   +0.08%
===========================================
  Files           36       34       -2
  Lines         1432     1506      +74
===========================================
+ Hits          1406     1480      +74
  Misses          26       26

Impacted Files
Coverage Δ

ppa/archive/views.py
97.34% <100%> (+1.09%)
:arrow_up:

ppa/archive/forms.py
100% <100%> (ø)
:arrow_up:

ppa/archive/admin.py
100% <100%> (ø)
:arrow_up:

ppa/archive/tests/test_views.py
100% <100%> (ø)
:arrow_up:

ppa/archive/tests/test_admin.py
100% <100%> (ø)
:arrow_up:

ppa/archive/urls.py
100% <100%> (ø)
:arrow_up:

ppa/archive/models.py
98.24% <0%> (-0.06%)
:arrow_down:

...hive/migrations/0003_add_description_collection.py

...on/migrations/0003_update_archive_manager_perms.py

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b54dbd...378a8dc. Read the comment docs.

@bwhicks after thinking about this more, I think you should test and try to make this handle adding all ~5000 items.  Even though that would be a silly use case for adding content to collections, it would give us good confidence that the logic will scale up (and if it doesn't and is difficult to scale up, then let's find out and document the threshold where it will fail, and handle appropriately in the interface).

@rlskoeser I've refactored with an approach that uses Django's session framework to store the DigitizedWork ids when passing them from the admin to the custom view. It also successfully handled the entire set of 5,000 works (with a few seconds delay). Now I'll turn to the naming and spot issues, but I think this will fit larger use cases than will likely be done with the bulk add.


Add Mutation for "Dropping a statement"

First, I'm very very impressed by Stryker, it brings so much value to see how robust your code / how well tested it really is. There's been much of badly written tests / insisting on having high test coverage (like a hard limit of 90% or whatever) which was always, in my opinion, very counterproductive to having well-maintained software. What's the best way to support your project (financially or otherwise ?)
Is your feature request related to a problem? Please describe.
Yes, I could say so.
In my code, there are some unused properties (or lines of code), that could go undetected by stryker.
Lets take a simple example
class myclasss {
   constructor(foo) {
        this.foo = foo;
   }
   add(a, b) {
      return a + b;
   }
}

// test file :

expect(new myclass("hello").add(10, 10)).to.equal(20);

Stryker will maybe show you that this.foo = "" is a surviving mutant (I'm not sure since I haven't tested it).
Since the codebase that I'm working on has multiple versions and the code has evolved quite a lot, there is a lot of such dead code that might not be detected by Striker. My test suite runs in under a second so I'm interested in having many many mutators to really "stress" my implementation.
Describe the solution you'd like
It could be great if mutants could simply drop a particular statement (one line statement maybe).
Maybe it would be complex because there are some statements that cannot be dropped without making the file invalid. For example if you drop a line containing a declaration const foo = "bar" and foo is used right afterwards, then of course, the test will fail.
Describe alternatives you've considered
Maybe there is a specific tool that would help to detect such issues that does not rely on mutation testing at all. Some kind of static analysis tool (I don't think eslint supports this kind of check).

Thanks for your kind words @edi9999 !
We're currently kind of reluctant to add new mutators. The reason is that all mutators are enabled by default and adding any might have a big impact on performance. That being said, there usually are mutants inside
We're currently investigating ways to introduce multiple "mutant levels". Level 1 would be fast and loose, level 3 would be very detailed and specific. We could then also allow you to run with "all" mutators active. A student is investigating this a.t.m. but it will take some time for results.

What's the best way to support your project (financially or otherwise ?)

Currently spreading the word is all we ask 💖. You're also welcome to contribute of course, our slack is a good place to start for that. I'll ask my colleagues if we would like to have donations/sponsorship.


Add slow refelection-based generic tensor arithmetic

Currently we have no fallback for a T that we don't understand in the Tensor arithmetic methods, we'll just throw.
We could instead construct a ITensorArithmetic that uses reflection to locate operators on that type.  This is slow for a couple reasons:

Built in types have operators that the JIT understands and replaces with instructions rather than method calls.
We have to use reflection to locate the operators.

Sometimes "really slow" can be better than not at all.  If that turns out to be the case then we can do this.
This may not even be in scope given what we decide for https://github.com/dotnet/corefxlab/issues/1798.
I could imagine this coming in scope if we decide computation is important, we're OK with slow computation, and we care about new primitive types (like System.Half) that the framework/JIT doesn't know about.

Dependent on #1798


Sui Code Bug or Feature Request Sui wallet not linking NFTs, taking fee

Steps to Reproduce Issue
Fill this in with the concrete steps needed to reproduce the bug. When providing code in the reproduction steps, use the smallest snippet of code that demonstrates the issue, removing any extraneous details.

Call function .
Use return value and call .

Expected Result
being able to link nfts to wallet taking fee but not storing nft
e.g.
Expected  to return 42.
Actual Result
Claiming nft rewards once  links to wallet takes fee  but once transaction processed just gos back to demo and nothing in sui wallet but minus balance please see attached photos
Specify what the actual unexpected outcome was.
e.g.
 returned 41.
System Information

OS: <specify OS vers

ion>
Compiler:
![20221028_091114](https://user-images.githubusercontent.com/116558527/198539219-1c24cd78-57d1-435e-917a-ba3554

c5ba89.jpg)

transaction details


Cannot rejailbreak after phone dying/shut off

Describe the bug
I had my jailbreak working perfectly fine, my phone dies, ive been trying for over 2 hours and had no luck rejailbreaking.
Im stuck at 1/29 exploiting kernel then it crashes
Ive tried:
airplane mode
checking for OTA update
disable daemons
wait 30 seconds and 2 minutes
restarting phone normally
To Reproduce
Steps to reproduce the behavior:

let your phone die
jailbreak

Screenshots
If applicable, add screenshots to help explain your problem.
Device (please complete the following information):

iOS Version: 13.3
iOS Device:  pro max
unc0ver Version: 4.0.2 and 4.0.3 (ive done both)

Place an "x" between the brackets if true:

[ x] this is a bug others will be able to reproduce
[ x] this issue is present with all tweaks uninstalled(except for default packages) or disabled
[ ] this issue is present after a rootfs restore
[x ] this issue is present on the latest version of unc0ver

Logs
If applicable, add logs or error messages here.

Duplicate of #1701

@cheerless Trying removing and reinstalling unc0ver. Not 100 percent sure it will work but its worth a try.


Add German Translation

Thanks!

I think, you have to rename the folder to "de"...

If you put this in the console
i18n.lng();

what does it tell you?

i18n.lng();

"de"

Huh, interesting.

Renamed, thanks.


Large in-memory S3 transfers

I've got a suite of applications that produce very large binary buffers that I'd like to be able to intermittently write to and read from S3 with.  I'd like to avoid large reallocations and copying of data and instead transfer the buffer directly to/from S3.
The PutObjectRequest wants an IOStream and the GetOjectRequest will allow you to provide a factory that creates an IOStream.  So the best workaround I've got so far is to create a StringStream and use pubsetbuf() to override the stream's underlying buffer (in the snippets below, I've got void* buffer and size_t bufferSize).
For the put:
    std::shared_ptr<Aws::StringStream> stream =
            Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG);

    stream->rdbuf()->pubsetbuf(static_cast<char*>(const_cast<void*>(buffer)),
                               bufferSize);
    stream->rdbuf()->pubseekpos(bufferSize);

    stream->seekg(0);

And for the get:
    request.SetResponseStreamFactory(
            [buffer, bufferSize]()
            {
                std::unique_ptr<Aws::StringStream>
                        stream(Aws::New<Aws::StringStream>(ALLOCATION_TAG));

                stream->rdbuf()->pubsetbuf(static_cast<char*>(buffer),
                                           bufferSize);

                return stream.release();
            });

This appears to work with the testing I've done so far, and as far as I understand streams it seems like this should accomplish what I want in terms of avoiding large allocations or copies.  Just wondering if you have any thoughts on an easier/better way to do this or have any plans to add any overrides in the API to make this a little easier.
Also, to confirm, a single PutObjectRequest/GetObjectRequest will automatically do multi-part uploads for large transfers right?  And if I don't override the rate limiter, by default it'll use as much network bandwidth as it can get?
Thanks.
-Adam

Have you looked at aws-sdk-cpp-transfer? It does many of the things you are
trying to do. It will allocate some buffers upfront, but afterwards will
just do copies. It will also do multipart for you automatically if the file
is over 5mb in size.
aws-sdk-cpp is a low-level client. Meaning, it is just an interface for
talking to the s3 service and nothing more.
What you have should work at first glance, hut the transfer manager is
designed for just this purpose.
Also, you are correct that if you dont override the rate limiter, then
everything will he wide open.
We discuss the transfer manager in the video in the readme. Its the last 15
minutes or so.
On Jan 3, 2016 7:49 AM, "Adam Sylvester"<EMAIL_ADDRESS>wrote:

I've got a suite of applications that produce very large binary buffers
that I'd like to be able to intermittently write to and read from S3 with.
I'd like to avoid large reallocations and copying of data and instead
transfer the buffer directly to/from S3.
The PutObjectRequest wants an IOStream and the GetOjectRequest will allow
you to provide a factory that creates an IOStream. So the best workaround
I've got so far is to create a StringStream and use pubsetbuf() to
override the stream's underlying buffer (in the snippets below, I've got void*
buffer and size_t bufferSize).
For the put:
std::shared_ptr<Aws::StringStream> stream =
        Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG);

stream->rdbuf()->pubsetbuf(static_cast<char*>(const_cast<void*>(buffer)),
                           bufferSize);
stream->rdbuf()->pubseekpos(bufferSize);

stream->seekg(0);

And for the get:
request.SetResponseStreamFactory(
        [buffer, bufferSize]()
        {
            std::unique_ptr<Aws::StringStream>
                    stream(Aws::New<Aws::StringStream>(ALLOCATION_TAG));

            stream->rdbuf()->pubsetbuf(static_cast<char*>(buffer),
                                       bufferSize);

            return stream.release();
        });

This appears to work with the testing I've done so far, and as far as I
understand streams it seems like this should accomplish what I want in
terms of avoiding large allocations or copies. Just wondering if you have
any thoughts on an easier/better way to do this or have any plans to add
any overrides in the API to make this a little easier.
Also, to confirm, a single PutObjectRequest/GetObjectRequest will
automatically do multi-part uploads for large transfers right? And if I
don't override the rate limiter, by default it'll use as much network
bandwidth as it can get?
Thanks.
-Adam
—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-cpp/issues/64.

Correction
aws-sdk-cpp-s3* is a low-level client.
On Jan 3, 2016 8:07 AM, "Jonathan Henson"<EMAIL_ADDRESS>wrote:

Have you looked at aws-sdk-cpp-transfer? It does many of the things you
are trying to do. It will allocate some buffers upfront, but afterwards
will just do copies. It will also do multipart for you automatically if the
file is over 5mb in size.
aws-sdk-cpp is a low-level client. Meaning, it is just an interface for
talking to the s3 service and nothing more.
What you have should work at first glance, hut the transfer manager is
designed for just this purpose.
Also, you are correct that if you dont override the rate limiter, then
everything will he wide open.
We discuss the transfer manager in the video in the readme. Its the last
15 minutes or so.
On Jan 3, 2016 7:49 AM, "Adam Sylvester"<EMAIL_ADDRESS>wrote:

I've got a suite of applications that produce very large binary buffers
that I'd like to be able to intermittently write to and read from S3 with.
I'd like to avoid large reallocations and copying of data and instead
transfer the buffer directly to/from S3.
The PutObjectRequest wants an IOStream and the GetOjectRequest will allow
you to provide a factory that creates an IOStream. So the best workaround
I've got so far is to create a StringStream and use pubsetbuf() to
override the stream's underlying buffer (in the snippets below, I've got void*
buffer and size_t bufferSize).
For the put:
std::shared_ptr<Aws::StringStream> stream =
        Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG);

stream->rdbuf()->pubsetbuf(static_cast<char*>(const_cast<void*>(buffer)),
                           bufferSize);
stream->rdbuf()->pubseekpos(bufferSize);

stream->seekg(0);

And for the get:
request.SetResponseStreamFactory(
        [buffer, bufferSize]()
        {
            std::unique_ptr<Aws::StringStream>
                    stream(Aws::New<Aws::StringStream>(ALLOCATION_TAG));

            stream->rdbuf()->pubsetbuf(static_cast<char*>(buffer),
                                       bufferSize);

            return stream.release();
        });

This appears to work with the testing I've done so far, and as far as I
understand streams it seems like this should accomplish what I want in
terms of avoiding large allocations or copies. Just wondering if you have
any thoughts on an easier/better way to do this or have any plans to add
any overrides in the API to make this a little easier.
Also, to confirm, a single PutObjectRequest/GetObjectRequest will
automatically do multi-part uploads for large transfers right? And if I
don't override the rate limiter, by default it'll use as much network
bandwidth as it can get?
Thanks.
-Adam
—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-cpp/issues/64.

Thanks - I hadn't noticed that.  Looking at TransferClient, I only see an interface for uploading and downloading files from disk by filename... am I missing an API that allows you to do this with in-memory buffers (i.e. I have a void* of a known length that I want to send directly to S3)?  That being said, even if TransferClient doesn't support this, I think it will be a very helpful example to help me fine-tune the code I'm working on.

have a look here and work backwards, you should be able to reuse our resource management interface.
https://github.com/awslabs/aws-sdk-cpp/blob/master/aws-cpp-sdk-transfer/include/aws/transfer/TransferClientDefs.h

Thanks for the pointer.  Does the C++ SDK support multipart downloads?  I see multipart upload requests in sdk-s3 and examples of them being used both in sdk-s3-integration-tests and sdk-transfer but don't see any reference to multipart downloads anywhere (or an option to specify this in the GetObjectRequest.
Am I just missing this?  If not, any guesses as to when the SDK will be updated to support it?

Each service client supports whatever a given service exposes, they are
generated directly from the service models. As to multi-part download,
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html
I recommend reading the api docs for s3. Once you determine what low-level
operations you want to do, you can then use the client to perform that
operation.
On Jan 10, 2016 8:46 AM, "Adam Sylvester"<EMAIL_ADDRESS>wrote:

Thanks for the pointer. Does the C++ SDK support multipart downloads? I
see multipart upload requests in sdk-s3 and examples of them being used
both in sdk-s3-integration-tests and sdk-transfer but don't see any
reference to multipart downloads anywhere (or an option to specify this in
the GetObjectRequest.
Am I just missing this? If not, any guesses as to when the SDK will be
updated to support it?
—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-cpp/issues/64#issuecomment-170366323.

Thanks... I should have taken a closer look.  I am successfully using the S3Client::SetRange() method and everything is working as expected.  In terms of potential future enhancements, it would be a little easier to use the SDK if it supported passing in some type of C++ range object rather than a string, but now that I know how this works it is easy enough for me to wrap this in my own method and build up the string under the hood.

In your example above with the StringStream for the "get" how do you access the underlying stream?
If I understand this pattern correctly:
Using the lambda you set the output to a StringStream using request.SetResponseStreamFactory.
Then call:
auto getObjectOutcome = Client->GetObject(request);
where the call to GetObject blocks. So how do you read the stream after that?
If it is I file I see I can read the file. But I am having trouble figuring out how to do it with a stream.
Any help would be appreciated.

So the cool thing is that you don't need to 'get' the underlying stream at all.  When the AWS SDK uses the StringStream to write, it's writing directly into the void* buffer that you provide to the lambda.  So, you've got the contents of the S3 object right in buffer... no need to get at the StringStream object itself.
To clarify a bit though, when I want the entire S3 object in memory, I first ask S3 the size of the object, then allocate a buffer / resize a vector to be that size, then pass that pointer to the lambda.  buffer has to be at least as large as the S3 object or something bad's going to happen (I guess I'm not sure offhand what'll happen if you did the pubsetbuf() with bufferSize < s3ObjectSize, but I am guessing either the StringStream write would block or something would index OOB).
I've been using this successfully so far but of course would like to get some more miles on it in case there's a shortcoming I haven't run into yet, so let me know if you have any issues with the approach.

Thanks for the quick reply. Your answer is really helpful.
This seems like it would work when one can read the entire object in memory.
I was hoping to find a way to stream large objects processing them as they are being read without waiting for the entire object to be read into memory. In this way it doesn't matter how big the object is and with a multi-threaded reader one can do parallel reads to several objects at once.
What this will do for me is reduce the latency of the request/response in a streaming server I am writing.
So in short I am trying to figure out how to attach to the underlying stream object so I can de-serialize the google protocol buffer objects being streamed.

Sure thing.
Yep - something similar to what you're saying is on my todo list too.  You can use S3Client::SetRange() to specify just a portion of the object that you want.  The C++ SDK is thread-safe.  So you could check up front what the object size is and either allocate one big buffer that all threads share or allocate a per-thread buffer at whatever size you want then kick off a pool of threads that read a desired range of the object (via SetRange() and the pubsetbuf() stuff above) and do whatever processing you need.  I haven't had a chance to implement this yet myself but I think all the pieces should play nice together.

Thanks for your help. I'll look into it. And report back if I have anything of value to add.
But I think I am solving a slightly different problem:
Is there a way to pass in a stream with dynamically allocated memory that the S3 api can write to while my thread reads from it?
I am thinking my server (acting as an s3 client) would be reading several large objects at once streaming them back while processing them and putting the result of the processing on separate dedicated queues that get drained in proper order by a sender thread. This sender thread would be sending responses back to my client.
So it's not necessarily breaking up a large object, but streaming it. And I  don't think it would be a problem if  the memory is dynamically allocated, that would actually be easier if the stream would handle it as basic IO streams do.
It seems to me the advantage of streaming (in general) is being able to process the head of an arbitrarily sized (and possibly huge) stream at the client side while the server is still sending the tail portion. Since I am processing packets out of the S3 object and sending a set of response messages  for each this would reduce the latency of the request/response between my client and server because I don't have to wait until I receive the entire object before sending response messages.
I guess I'm stuck on the notion of reading the object as a dynamic stream. I don't mind the memory allocations.

Ahh gotcha... in that case, you should be able to just send the SDK a normal StringStream (without the pubsetbuf()).  The SDK would write to the stream and the StringStream would internally dynamically allocate more space as needed.  You want to be able to access the stream afterwards, so you create the stream beforehand and make a lambda for SetResponseStreamFactory that just returns a pointer to that stream.  You keep the stream in scope and then just access it directly after the SDK call.

That is what I have been trying to do but to no avail.
here is the compiler output:
../src/mnem_extract_server.cpp:197:17: error: no matching function for call to ‘Aws::S3::Model::GetObjectRequest::SetResponseStreamFactory(AwsS3DataMoverPub::processPktFile()::__lambda4)’
                });```

For the following code:

This is defined in the class protected data:
```std::shared_ptr<Aws::StringStream> s3_stream;```

 This is in the Ctor of the class:

```s3_stream = Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG);```

And finally this is the call to the lambda using a local scope variable:

    std::shared_ptr<Aws::StringStream> m_stream = s3_stream;
    getObjectRequest.SetResponseStreamFactory(
    					      [m_stream,obj_name]()
    					      {
     						cout << obj_name << " - lambda called with stream" << endl;
    						return m_stream;
    					      });

I just skimmed over this, but take a look at the response stream factory signature... I'm pretty sure it wants a raw rather than shared pointer (so change to return m_stream.get()).

Thanks!
I really appreciate the time you are taking to help me out on this.
I'm wasn't sure if this needed to be a shared pointer or not. I was following the pattern from the other stuff. I'm new to the smart pointer concept.
The following code compiles but is it ok at runtime?
I assume I own the stream but how do I know the api thread that is using it is done? What if I want to implement a "cancel" ? It seems that a smart pointer might be needed in this case.
I'm surprised the factory signature uses a raw pointer:
The lamda call:
    getObjectRequest.SetResponseStreamFactory([m_stream]()
                                            {
                                               return m_stream;
                                            });```

In the Ctor:
``` s3_stream = new Aws::StringStream(ALLOCATION_TAG);```

Where s3_stream was defined in protected data as:

```Aws::StringStream* s3_stream;```

The memory returned by the reaponsestreamfactory will be deleted when the
result object goes out of scope. Don't pass underlying smart pointer memory
to it.
On Jan 14, 2016 4:39 AM, "Adam Sylvester"<EMAIL_ADDRESS>wrote:

I just skimmed over this, but take a look at the response stream factory
signature... I'm pretty sure it wants a raw rather than shared pointer (so
change to return m_stream.get()).
—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-cpp/issues/64#issuecomment-171633275.

@JonathanHenson
A couple of questions:
From the examples I don't see a result object. Is that visible to the user? i.e. can I control the scope.  Or are you saying this is a bad idea?
From the example:
GetObjectRequest getObjectRequest;
getObjectRequest.SetBucket(fullBucketName);
getObjectRequest.SetKey(keyName);
getObjectRequest.SetResponseStreamFactory([](){
   return Aws::New<Aws::FStream>(ALLOCATION_TAG,DOWNLOADED_FILENAME, std::ios_base::out );
                                 });
auto getObjectOutcome = s3Client->GetObject(getObjectRequest);

If I understand this correctly GetObject blocks until the transfer is complete.
Is there anyway to interface to this API asynchronously so I can initiate a transfer into a stream and in a separate thread read data out of the stream? This way I can process data immediatly

Result object is inside the outcome. e.g. getObjectOutcome.GetResult();
You can take ownership of the Result via move semantics. In that scenario, the memory is yours once you call GetResultWithOwnership();
All low-level api calls have async and future variants. So you can always use those.

I can't get this to work... I have a char* array with some eof characters (.png file, but I would like not use FStream), I can't figure out how to send it. I can't use StringStream, because of eof. The solution that was originally suggested by OP doesn't work for me, I just get timeout error.
I've also tried creating IOStream like described here: http://stackoverflow.com/questions/7781898/get-an-istream-from-a-char
But that doesn't work either.

This is currently being reviewed internally, but I'd love your thoughts while we're at it:
https://s3.amazonaws.com/aws-sdk-cpp-demos/transferRewrite.diff
Notes:
Deleted TransferClient and everything related to it. Added a simpler interface, TransferManager. TransferHandle is the other half of the control interface which is what the user would use to control their transfers. I added a few utilities to core to support the changes. Largely a generic queue locker, and a std::stream_buf that runs off of a preallocated buffer.
Later I'll be adding parallel downloads (harder to pull off), and directory synchronization, but this has feature parity with the original TransferClient.
Major performance differences:
All stream memory is preallocated (so no more allocation during the transfer)
Threading model is simplified.
No busy waits
Things I'm not happy about:
Mutex heavy
I discovered that we aren't handling Http/Continue correctly on Windows while testing. however, this isn't a regression from the previous code base so I'll add it to the backlog.
For multi-part uploads, we only update the transfer progress for each part because it was too complicated to back out updates if a part fails.
Testing notes:
all unit tests, and integration tests pass.
Uploads saturate the nic (which was my goal)
Downloads, since they aren't in parallel, still need work from a performance standpoint.

S3Client has no member named "SetRange", right?
error: 'class Aws::S3::S3Client' has no member named 'SetRange'
  context->client->SetRange(range.str());
                   ^

See GetObjectRequest::SetRange
Sent from my iPhone

On Dec 6, 2016, at 7:52 AM, hailbird<EMAIL_ADDRESS>wrote:
S3Client has no member named "SetRange", right?
error: 'class Aws::S3::S3Client' has no member named 'SetRange'
context->client->SetRange(range.str());
^
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I tried it before I asked the question, but failed.  See [https://github.com/aws/aws-sdk-cpp/issues/366] for detail.  I checked the signature is correct by enhancing libs3 and do the same thing.

Hi,
In advance sorry if this is not the right thread for my question.
I have a question about TransferManager Upload file function. I am able to transfer 5Mb file, but when I tried to upload biger file 7MB or more. I get "[WARN] 2017-01-20 12:51:30 AWSErrorMarshaller [72860] Encountered Unknown AWSError
XML_ERROR_MISMATCHED_ELEMENT
Unable to generate a proper httpResponse from the response stream.   Response code: 503:" error.
What does this error means?

I am having similar problems with the AwsTransfer library, particularly on the "DownloadFile" function. Ideally, I do not want to write to a file and then have to read the contents back into memory. Your integration tests here: https://github.com/aws/aws-sdk-cpp/blob/master/aws-cpp-sdk-transfer-tests/TransferTests.cpp#L194-L235 check the uploaded file by downloading to an FStream. I am able to successfully do this; however, I have not been able to successfully implement the this to an std::stringstream. For small transfers (less than 20MB) I don't seem to have problems, but when I increase the size to 21MB, I no longer get the entire file. Here is my attempt:

std::shared_ptr<std::stringstream> AwsTransfer::SyncDownload(const std::string& bucket, const std::string& key) {
    std::shared_ptr<std::stringstream> contents = std::make_shared<std::stringstream>();
    std::shared_ptr<TransferHandle> request = m_transfer->DownloadFile(bucket, key, [&contents]() {
        std::iostream* out = Aws::New<std::iostream>(allocationTag, contents->rdbuf());
        return out;
    });

    request->WaitUntilFinished();

    return contents;
}

Do I need to preallocate the size of the stringstream before passing?

The answer to the original question isn't correct or helpful (ie, use transfer manager).   In my experience, you cannot use  stringstream to upload binary data; at least, not in the way documented above, with pubsetbuf, pubseekpos and seekg(0).
Use boost bufferstream.   See https://github.com/aws/aws-sdk-cpp/issues/533

we're about to kill of std::iostream usage in favor of a simpler interface. Iostream will be supported but wrapped.    Seems like people were still having same problem in January 18.   See #785.

I've got a suite of applications that produce very large binary buffers that I'd like to be able to intermittently write to and read from S3 with. I'd like to avoid large reallocations and copying of data and instead transfer the buffer directly to/from S3.
The PutObjectRequest wants an IOStream and the GetOjectRequest will allow you to provide a factory that creates an IOStream. So the best workaround I've got so far is to create a StringStream and use pubsetbuf() to override the stream's underlying buffer (in the snippets below, I've got void* buffer and size_t bufferSize).
For the put:
    std::shared_ptr<Aws::StringStream> stream =
            Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG);

    stream->rdbuf()->pubsetbuf(static_cast<char*>(const_cast<void*>(buffer)),
                               bufferSize);
    stream->rdbuf()->pubseekpos(bufferSize);

    stream->seekg(0);

And for the get:
    request.SetResponseStreamFactory(
            [buffer, bufferSize]()
            {
                std::unique_ptr<Aws::StringStream>
                        stream(Aws::New<Aws::StringStream>(ALLOCATION_TAG));

                stream->rdbuf()->pubsetbuf(static_cast<char*>(buffer),
                                           bufferSize);

                return stream.release();
            });

This appears to work with the testing I've done so far, and as far as I understand streams it seems like this should accomplish what I want in terms of avoiding large allocations or copies. Just wondering if you have any thoughts on an easier/better way to do this or have any plans to add any overrides in the API to make this a little easier.
Also, to confirm, a single PutObjectRequest/GetObjectRequest will automatically do multi-part uploads for large transfers right? And if I don't override the rate limiter, by default it'll use as much network bandwidth as it can get?
Thanks.
-Adam
I opened another issue that is similar to this -
https://github.com/aws/aws-sdk-cpp/issues/1351

Having hard time figuring this out, any help is appreciated.
I tried a single put object with in-memory buffer as below. But when its uploaded to S3 and I manually download the file from S3 console to cross check validity of the content(simple text file), it does not contain new line feeds '\n' or '\r'.
Pretty much implemented some parts of TransferManager.cpp for SinglePartUpload(), except that source file read is proprietary filesystem.
static const char CLASS_TAG[] = "EventHeader"; auto streamBuf = Aws::NewAws::Utils::Stream::PreallocatedStreamBuf(CLASS_TAG, buffer, static_cast<size_t>(bytesRead)); auto preallocatedStreamReader = Aws::MakeSharedAws::IOStream(CLASS_TAG, streamBuf); object_request.SetBody(preallocatedStreamReader); object_request.SetContentType("text/plain"); // tried "binary/octet-stream"
Hexdump of the buffer that I passed to streambuf.
data length: 21
00000000 61 61 61 61 61 61 0a 62 62 62 62 62 62 0a 63 63 |aaaaaa.bbbbbb.cc|
00000010 63 63 63 63 0a |cccc. |
'0a' is a line feed.
What am I missing?


@urql/svelte query doesn't return a promise

@urql/svelte query doesn't return a promise. According to forums and the docs there should be a toPromise() method which does not exist for query. I've also read there is a hack to use mutations, which also doesn't work in urql/svelte. The way it's working now is very unpleasant. The query writes to a store and then the app waits for the store to retrieve data. This seems to be a common problem but I've not found an acceptable solution.
urql version & exchanges:
"@urql/svelte": "^1.3.3"
"urql": "^2.2.0"
Steps to reproduce
query(exampleQuery)
  .then((result) => {
   console.log(result);
  });

Expected behavior
A query within a function should be able to wait for fetching to be false before returning.
Actual behavior
The function returns the query with fetching true, and a promise does not work in any form.

I believe you may have gotten two APIs confused. You can find the exact API details on two different docs pages:

Svelte: https://formidable.com/open-source/urql/docs/api/svelte/#query
Core: https://formidable.com/open-source/urql/docs/api/core/#clientquery


Refactored footer icons + many new icons

sorted icons alphabetically
social icons for Mastodon, Matrix and Telegram
add rel="me" to social links

rel="me" is a de facto standard for this kind of links
check: https://microformats.org/wiki/rel-me
FontAwesome is missing an icon for Matrix. Request for inclusion was opened in 2018 but I guess they don't want to add an icon without Matrix.org paying the comission.

I had refactored the footer icons, which will break configuration, but make icons easier to add and modify besides making the code cleaner with all the data on _data.
One option to improve it further is to move user data to _config,yml separating it from the icon 'templates' data.

Calendly
Google Scholar
Map (OpenStreetMap)
Medium
Patreon
Phone
SMS
Twitch
VK

What is Matrix mentioned in the first comment?

What is Matrix mentioned in the first comment?

It is a decentralized real-time chat protocol. It has many clients like Element and Fluffy.  Although it is no so decentralized yet as most users and servers are located on Matrix.org namespace.

So, is it a chat app server?

So, is it a chat app server?

Protocol. Has many clients, servers/bridges.

So, why do you need to add Matrix? You could possibly add Element rather than Matrix, right?

So, why do you need to add Matrix? You could possibly add Element rather than Matrix, right?

I can use my account with any client and connect to networks like IRC, Telegram and Discord giving that they have bridges. That is my Matrix address on matrix.org. Element is just one client.

Well, there's good AND bad in it. Goodis that people can contact you on bug info's directly instead of issues. Bad is that, there might be spam messages.

And spam is irritating.

And I am now interested in Element

Well, Matrix is new here.

I got an idea. Why not you get Matrix's icon, customize it and use it here?

I got an idea. Why not you get Matrix's icon, customize it and use it here?

Adding exceptions make the code less maintainable and it's not critical to have. I used a close substitute.

Ok

From what I see, it will be a breaking change since you'll have to update one yaml or another

Yes

It is now merged, I made some change to make it backward compatible 👍
Thanks for the PR @danisztls that's a whole lot of new icons we have now ehehe


Not redirecting team members profile to Linkedin when clicked on picture and name

When we click on an image or name in the Our Team Members section, it does not redirect to the linkedin profile. it only redirects to the LinkedIn profile when click on LinkedIn image.
I would like to make it such a way that when clicked on the image and name it redirects to the linkedin profile.
I would like to work on this issue please assign it to me.

That's intensional @Yasshu55 , we have provided below a LinkedIn icon for better understanding as In mobile view user can slide left or right to see more team members.

That's intensional @Yasshu55 , we have provided below a LinkedIn icon for better understanding as In mobile view user can slide left or right to see more team members.

Ohk then it's fine...but my code does not affect how it looks in pc or mobile or any view. It looks the same, but when clicked on the name or image instead of the icon even then it redirects. As many times people on various screen sizes click on the image or name to open the profile.


fix demo notebook in docs

model demonstration notebook was not shown in the docs

Codecov Report

Merging #191 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #191   +/-   ##
=======================================
  Coverage   86.03%   86.03%
=======================================
  Files          30       30
  Lines        1840     1840
=======================================
  Hits         1583     1583
  Misses        257      257

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd84777...9d0aed4. Read the comment docs.


Add Observable.mapAsyncOrdered for parallel mapping over an observable

I've briefly discussed this on gitter before, but can't find a search feature in gitter to keep track of the conversation so moving to github.
I'm already making use of the awesome Observable.mapAsync but would love it if it would preserve the order of the elements.
Here is a use case as to why I'd like this operator.
I'm processing a large dataset which can potentially fail.  As the processing progresses, I'd like to periodically output the id of the record which was last successfully processed.  If the processing fails, I can then continue from the last id that was successfully processed (the source of my data provides the ability to resume from a known point).
If the Observable is using mapAsync, the ordering is no longer preserved and I can't easily record what the id of the last processed record is.
If ordered parallel operations is painful, I'd be open to other patterns or suggestions to handle.
And for reference, this is the equivalent functionality in akka-streams
http://doc.akka.io/docs/akka/2.4/scala/stream/stages-overview.html#mapAsync (The default in akka is order preserving).

mapAsyncUnordered as akka?

@lukestephenson I understand the use-case. Well, it's a little difficult to implement. Any idea on the semantics involved? For example, the Akka implementation is it equivalent with the following code and can you check?
observable.bufferIntrospective(parallelism).flatMap { batch =>
  // Processing batch in parallel
  val task = Task.gather(batch.map(f))
  Observable.fromTask(task).flatMap(Observable.fromIterable)
}

This is similar to the approach highlighted here: https://monix.io/docs/2x/tutorials/parallelism.html#imposing-a-parallelism-limit
What this does is to build a source that emits batches and then process a batch in parallel, one by one. Preserving order in this case is OK, because you only care about a finite batch at a time.
However, it has a downside: if one of those batches triggers a slow Task that takes a long time to finish, it will block pending batches. So if you have say parallelism = 10, what this means is that this will be a maximum limit, instead of actual active workers, because due to uneven execution times you can end up with fewer tasks being active at any given moment.
But this kind of goes with the territory of wanting ordering - because we can't do unlimited buffering, regardless of implementation, at some point the implementation will have to freeze progress in order to wait on some slow Task to finish.
Does this make sense? Is this acceptable?

@hepin1989 our current implementation is unordered because IMO it's the more useful variant. We can add a mapAsyncOrdered if there's interest.

That's okay,just need a mind shift here :)

@alexandru Thanks for the suggestion
I don't think bufferIntrospective will work in my case.  I'm expecting the publishing of elements to be slower than the consumption, so I don't think it would back pressure enough to cause the batch processing piece to occur in parallel.  That said, I could use bufferTumbling to obtain a batch for processing. In the interests of simplicity, I'm happy to go with this approach.
With regard to the operator if we decide to implement.
Waiting on a slow task at some point is acceptable (and can't be avoided to guarantee ordering).  A buffer which is larger than the level of parallelism can in most cases ensure the desired level of parallelism is achieved if a few tasks finish out of order internally.
I took a look at the akka implementation - https://github.com/akka/akka/blob/5edead2bda89e5c1bcaa39ae17578d6d86fa03ed/akka-stream/src/main/scala/akka/stream/impl/fusing/Ops.scala#L1138.
For each task that is executed in parallel, it is first added to a Buffer, after first being wrapped in an object Holder which tracks if the task as finished.  When a task finishes, it updates it's Holder to flag completion.  Then it can peek at the head of the Buffer and if the head is completed, it can push that onto the next stage (and continue through the Buffer until the head is no longer a completed task).
I'm considering trying to implement and submit a PR, but this may not be the easiest introduction to monix internals.  I first wanted understand the existing MapAsyncParallelObservable, and am currently at the point of trying to understand why it uses a AsyncSemaphore rather than a TaskSemaphore (because it appears to then reimplement some of the logic of TaskSemaphore).  But that's a separate discussion.

@lukestephenson ah, very interesting.
So one possible and simple implementation would be to use a buffer onto which we push Future references (or something similar) and then on the other side we keep waiting on the head to be completed in order to signal it downstream.
Yes, it's not exactly an easy operator to implement. You can usually start with something that works but that is not that well optimized and then work your way from there though pain and sheer will :-)
source.map(x => task(x).runAsync) // yields Futures
  .whileBusyBuffer(BackPressure(parallelism))
  .mapFuture(f => f)

In the meantime, speaking of that substitute, make sure you read: https://monix.io/docs/2x/tutorials/parallelism.html

@lukestephenson so you can try your hand at a PR if you think you can do it. I'll gladly help you and review it, just as a word of warning, when working with concurrency, there's some pain involved, you won't get it correct from the start.
Or if you think you'd like some simpler operator, open other tickets 😄

I am now also in a position where I need a mapAsync that preserves the order. Is there any ETA / update regarding it?

I never got around to attempting it.  Went with one of the workaround suggested by @alexandru

That workaround should be pretty OK too, it's what the logic should be anyway, although in a specialized implementation we might be able to eliminate some overhead.

I'm also interested on such operator. Here is the pattern I'm using right now:
val semaphore = AsyncSemaphore(maxParallelism)

source
  .map(x => semaphore.greenLight(() => run(x).runAsync))
  .asyncBoundary(OverflowStrategy.BackPressure(maxBufferSize))
  .mapFuture(identity)

I've experienced a better performance overall, with the cost of more memory when there is a slow future on the stream.

+1 for needing a mapAsync preserving order


Passing a proc/lambda to group_label_method

The :label_method option in a grouped select supports receiving a proc/lambda instead of just a symbol. However, :group_label_method does not. It throws a type error. Any way to work around this?

I don't really remember off the top of my head, sorry, but I'd say it might be more something that Rails would/should provide instead. In any case, please use the mailing list or stack overflow for questions, we reserve the issues tracker for issues. Thanks!

Sorry, I did not put here as a question. I just thought it was an actual issue with this library, that could perhaps be solved within it. I'm sorry if it sounded as a question or me seeking help/support.

It seemed so, but maybe it was not and I interpreted it wrong. In any case, I'm not sure passing a proc is possible right now, but maybe we should add that support directly to Rails instead?


Disable NAM in the asset dropdowns + The shield button on NAM in the overview in phases 2/3/4

Currently we are checking for this with asset.display === "nam"
We should have a consistent way to check for NAM asset in the whole Namadillo.
A suggestion is to use the isNamAsset from https://github.com/anoma/namada-interface/pull/1292 and deleting the namadaAsset to avoid inconsistencies across chains


[DO NOT MERGE] MC LG FIX

No linked issues found. Please add the corresponding issues in the pull request description.  Use GitHub automation to close the issue when a PR is merged


package.json Update handling

Generally, I know there is no need to update and be always on the latest package updates. But I think it can only make things better to be update to date and get so the latest features of the packages. My question is how is the update handling in VS Code? Can I just make a PR and update these and they will be merged if the CI will succeed?
If that's the case we could think about adding e.g. the greenkeeper app or renovateapp which automatically makes PRs to this repository with the latest updates of the packages. (Changelog of the packages included)
Here's an overview about the latest output of npm-check-updates to see the actual situation:
 applicationinsights     0.17.1  →    1.0.0
 getmac                   1.0.7  →    1.2.1
 http-proxy-agent         0.2.7  →    2.0.0
 https-proxy-agent        0.3.6  →    2.1.0
 semver                   4.3.6  →    5.4.1
 yauzl                    2.8.0  →    2.9.1
 @types/mocha            2.2.39  →   2.2.44
 @types/sinon           1.16.34  →    4.0.0
 azure-storage           ^0.3.1  →   ^2.6.0
 clean-css                3.4.6  →    4.1.9
 coveralls             ^2.11.11  →   ^3.0.0
 cson-parser             ^1.3.3  →   ^2.0.0
 eslint                  ^3.4.0  →  ^4.12.0
 glob                   ^5.0.13  →   ^7.1.2
 gulp-azure-storage      ^0.7.0  →   ^0.8.0
 gulp-eslint             ^3.0.1  →   ^4.0.0
 gulp-filter             ^3.0.0  →   ^5.0.1
 gulp-image-resize      ^0.10.0  →  ^0.13.0
 gulp-mocha              ^2.1.3  →   ^4.3.1
 gulp-replace            ^0.5.4  →   ^0.6.1
 gulp-shell              ^0.5.2  →   ^0.6.3
 gulp-sourcemaps        ^1.11.0  →   ^2.6.1
 gulp-vinyl-zip          ^1.2.2  →   ^2.1.0
 husky                  ^0.13.1  →  ^0.14.3
 istanbul               ^0.3.17  →   ^0.4.5
 lazy.js                 ^0.4.2  →   ^0.5.0
 mime                    1.2.11  →    1.6.0
 minimatch              ^2.0.10  →   ^3.0.4
 mocha                   ^2.2.5  →   ^4.0.1
 optimist                 0.3.5  →    0.6.1
 queue                    3.0.6  →    4.4.2
 remap-istanbul          ^0.6.4  →   ^0.9.5
 sinon                  ^1.17.2  →   ^4.1.2
 source-map              ^0.4.4  →   ^0.6.1
 typescript               2.6.1  →    2.6.2
 typescript-formatter     4.0.1  →    7.0.0
 vinyl                   ^0.4.5  →   ^2.1.0
The following dependencies are satisfied by their declared version range, but the installed versions are behind. You can install the latest versions without modifying your package file by using npm update. If you want to update the dependencies in your package file anyway, run ncu -a.
 pump             ^1.0.1  →  ^1.0.3
 rimraf           ^2.2.8  →  ^2.6.2
 uglify-es       ^3.0.18  →  ^3.2.0
 vscode-nls-dev   ^2.0.1  →  ^2.1.6

We update our dependencies every now and then. They bring good stuff, but they also bring adoption work and possible bugs. Closing this, since this task is a periodic one.


Compiler: fix is_a? from generic class against generic class instance type

Fixes #12304

Yup, a commutstive rule was missing.
I'll later check if any other is missing.
I'll merge this now because @cyangle is really good at reducing interpreter bugs, so the more bugs we have fixed on master the less false positives they'll have.


⚠ Processors has degraded performance

In b640815, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 91 ms

Resolved: Processors performance has improved in bdc2f4e.


ISPN-11790 Deprecate CacheQuery

https://issues.redhat.com/browse/ISPN-11790
Depends on #8303

Ready for review @anistor

@gustavonalle I just noticed you have 2 similarly named ISPN-11790_deprecate_cache_query banches.

I got rid of it @anistor

Merged in master. Thanks @gustavonalle !


Update reset-physicaldisk.md

Fixed typo in 2nd example, corrected a potential error (i.e. if more than one bad disk is returned), and changed the example to use a 'safer' set of parameters.
Also changed code fencing to Powershell
This PR relates to issue #494

Good job with the for loop, to make sure all are reset. @doctordns your expertise is much appreciated, thanks for sharing with community!


Fix the link to 'Programming the RP2040'

Thank you, greatly appreciated! External changes (websites referenced, toolchain versions) also need the docs fixed...


BackwardsCompatibleAsyncPackage project does not build

Installed products

Visual Studio: 2017 & 2019

Description
BackwardsCompatibleAsyncPackage project does not build in VS2017 or VS2019.
Steps to recreate

Install VS2017 with extension development workload
Clone the VSSDK-Extensibility-Samples repo from GitHub
Open the BackwardsCompatibleAsyncPackage solution file in VS2017
Build the project

Current behavior
Build fails with these warnings & errors:
1>C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3245: Could not resolve this reference. Could not locate the assembly "Microsoft.VisualStudio.Shell.12.0". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.
1>C:\Temp\VSSDK-Extensibility-Samples\Backwards_Compatible_AsyncPackage_2013\BackwardsCompatibleAsyncPackage\BackwardsCompatibleAsyncPackage.cs(18,59,18,66): error CS0246: The type or namespace name 'Package' could not be found (are you missing a using directive or an assembly reference?)
1>C:\Temp\VSSDK-Extensibility-Samples\Backwards_Compatible_AsyncPackage_2013\BackwardsCompatibleAsyncPackage\BackwardsCompatibleAsyncPackage.cs(17,65,17,76): error CS0103: The name 'VSConstants' does not exist in the current context
1>C:\Temp\VSSDK-Extensibility-Samples\Backwards_Compatible_AsyncPackage_2013\BackwardsCompatibleAsyncPackage\BackwardsCompatibleAsyncPackage.cs(28,33,28,43): error CS0115: 'BackwardsCompatibleAsyncPackage.Initialize()': no suitable method found to override
Expected behavior
Project should build successfully and produce a VSIX file.

Never mind.  I didn't have VS2013 installed, which is clearly listed as a requirement in the readme for this project.  After installing VS2013 the project builds fine (in both VS2017 and VS2019).


Azure IKEv2 fragmentation issue

When azure IKEV2 gateway receives the fragmented UDP packets in out of sequence, it drops the packets. Its expecting us to send the packets in sequence. This is not possible in many scenarios, please fix this bug also please point me to any document which mentions this limitation. Please find the attachment

for more info.

@vishnubraj , Thank you for reaching out.
We are aware of this issue. Here is the Public Documentation on the recommended practices with setting up MTU on your device.
Doc: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-tcpip-performance-tuning
Will raise this to the Product Group and once we get an update will update the documentation.


[FEA] Transparent fallback to CPU reader when input codec not supported

Is your feature request related to a problem? Please describe.
When reading Parquet or ORC the compression codec is not specified by the user but rather detected in the individual files that are loaded.  If a codec not supported by cudf's I/O loaders is being used by one or more input files then the load will crash, failing the job.  Because the codec is not known at query planning time, the incompatible codec is detected well after the query has been planned to use the GPU for loading.  It would be nice if an unsupported codec for a file was detected by the plugin reader before calling cudf and transparently fallback to the CPU loader for that file.
Describe the solution you'd like
Currently the plugin is already accessing the footer of each file on the CPU, so we should be able to detect incompatible codecs before calling cudf.  At that point we would need to call into the CPU loading code to handle the file, convert the data to cudf columns and load on the GPU.

Duplicate of #5461


Segmentation results of face hair and skin

Hi,
I run the segmentation model of mmfashion.
But I found the results do not contain the face hair and skin segmentation results.
It is possible to obtain these results with the provided model.
Thanks!

@RenYurui  hi , how you tried it ? thanks in advance


cmake configure succeeds, but the build later fails: fatal error: 'TaskScheduler.h' file not found

[x] Ask for help on discord
[x] Consider providing a dump file using b2World::Dump

The file TaskScheduler.h isn't included into the project, and it also isn't searched by cmake.
Version: 3.0.0
clang-18
FreeBSD 14.1

Are you trying to build just the library or the samples/tests/benchmark apps? Only the latter make use of enkiTS.


Should split_job_s2grid and split_job_hex return original geometry instead of Points?

Currently both job splitters use the centroid of the geometry to determine which tile/hex they belong to. However, after the geometries are split per tile/hex, they aren't converted back to the original geometry, but instead kept as Points.
Would it be useful to return the original geometries instead of their centroids after the job splitting?

I think by default the original polygons should be returned, but we could provide an argument to use the centrods instead


Mir audit: Batchfetcher and mempool modules update to DSL codegen.

Builds upon #336

Added a fix for #353 , the fix may interfere with mpdsl.UponRequestTransactions if that call is for a number of txIDs that are not in memory anymore, but I could not find a single event being created for this purpose.

Also, found #352 along the way. We should discuss how much of an issue we think that could be (specially with the latest's availability module #336 .

Forced push to account for possibility of multiple client's requests with the same txID when fixing #353.

Added support for map types to codegen, although it is not being used at the moment. The mir.type field must define both key and val types in the following way keytype|valtype

I'm putting this on hold until #336 is polished and merged, let's have a look at this one afterwards.

Thanks a lot @xosmig for the comments, it's very helpful!

Thanks @xosmig and @matejpavlovic for the comments! :)


learnworlds.com

On the bright side. You may add learnworlds.com to the list of supported LMS systems.
There was some issues, but they were fixed. I've checked, now everything works.

Closing, will add it to exporter's project


Add revs limit for docs passed to filter functions

The existing revs_limit logic only accounts for depth of individual
branches, and ignores the case where you have a lot of branches, for
instance when a doc is heavily conflicted. This patch truncates the list
of revs when passing docs to the JS filter functions, because given
enough revs this can cause the couchjs processes to explode.

hrm, when filtering is there any point passing the _rev or rev history at all? (cc @davisp @kocolosk).

@chewbranca Does this also affects on other design functions? IIRC shows also receives documents with the whole _revisions list by default now. Not sure about update ones, never was curious about.
Another question is why filter function receives list of _revisions now? What the use case of this and may be it's better to not send it at all?

@rnewson @kxepal When I first saw this my first thought was to remove _revisions entirely assuming that it was basically an accident to have included it in the first place. But given that its there and its theoretically possible someone might rely on it I instead suggested the truncation approach since revs_limit exists so anyone relying on it can't correctly rely on a specific length of the _revisions list.

@davisp I'm not sure about someone to relay on it. What's practical use case for them in filter function? String all docs which has more than N revisions in history? Sounds strange.

hrm, fair enough, we can't be sure no one is depending on it.
+1

@kxepal I haven't got a clue why someone might use it. But it exists so I'm guessing someone somewhere would be angry if we removed it.

@davisp well, good point. Let's land this and run some poll later to deprecate and remove this.
However, we also need more general solution since here we improved filtered changes feed performance a lot, but still I can blow your query server with show and update function using the same vector. Having filter_revs_limit, shows_revs_limit, update_revs_limit config options is a bit weird.

+1. @chewbranca merge pls

@rnewson comment wasn't addressed.

which comment?

+1

after appropriate squashing

Merged in master.

Merged with https://github.com/apache/couchdb-couch/commit/eb60e41ad0afdad1281455e19cf1bf02419864d4 .

@chewbranca how come this isn't marked 'merged' like usual?

@rnewson I forgot to push back out the branch after I rebased my changes on top of master, so the sha's were different in master than here.

@rnewson Commit hashes in PR and on master are different. And merge commit doesn't contains magic works "This closes #107". One of these conditions need to be fixed to help ASFBot get to know which PR need to close/merge.

@chewbranca @kxepal it's certainly possible to do this 'right', can we be more careful for future PR's please.

@rnewsown Sure. Actually just need to always push changes to your feature branch before do the actual merge. If we'll ask for / do rebase against current master before all of that - would be awesome and this help us to avoid useless merge commits for single commit changes.


⚠ api.piste.gouv.fr healthcheck has degraded performance

In 81a920c, api.piste.gouv.fr healthcheck (https://api.piste.gouv.fr/cassation/judilibre/v1.0/healthcheck) experienced degraded performance:

HTTP code: 200
Response time: 3070 ms

Resolved: api.piste.gouv.fr healthcheck performance has improved in 904f5e7.


supervisor couldn't start logmonitor

Can't run container on armv7 arch, getting:

$ docker run jlesage/baseimage:ubuntu-18.04-v3.0.0-pre.7 /init
...
[init        ] giving control to process supervisor.
[supervisor  ] loading services...
[supervisor  ] loading service 'default'...
[supervisor  ] could not load 'logmonitor.dep': command failed (126)

$ docker version
Client:
Version:      18.03.1-ce
API version:  1.37
Go version:   go1.9.2
Git commit:   9ee9f40
Built:        Thu Apr 26 07:13:21 2018
OS/Arch:      linux/arm
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version:      18.03.1-ce
API version:  1.37 (minimum version 1.12)
Go version:   go1.9.5
Git commit:   9ee9f40
Built:        Thu Apr 26 07:46:45 2018
OS/Arch:      linux/arm
Experimental: false

The current version of the baseimage does not support ARM.
The next version will be muti-arch.

It's sad, thanx for info


Caching buildtrees only works with BuildElement (cannot enter a shell after a failed build)

See original issue on GitLab
In GitLab by [Gitlab user @abderrahimk] on Feb 15, 2020, 14:16
Summary
When I try to enter a shell on a build failure, I get an error instead.
Steps to reproduce

default configuration of buildstream
try to build an element that fails
enter s at the prompt

What is the current bug behavior?
Build failure on element: vm/repo.bst

Choose one of the following options:
  (c)ontinue  - Continue queueing jobs as much as possible
  (q)uit      - Exit after all ongoing jobs complete
  (t)erminate - Terminate any ongoing jobs and exit
  (r)etry     - Retry this job
  (l)og       - View the full log file
  (s)hell     - Drop into a shell in the failed build sandbox

Pressing ^C will terminate jobs and exit

Choice: [continue]: s

Dropping into an interactive shell in the failed build sandbox

Error while attempting to create interactive shell: Buildtree is not cached locally or in available remotes

What is the expected correct behavior?
a shell into the failed build.
Other relevant information

BuildStream version affected: 0591b0c8b038aa9f054b295748f3423384711d50

In GitLab by [Gitlab user @abderrahimk] on Feb 15, 2020, 14:16
changed title from {-c-}annot enter a shell after a failed build to {+C+}annot enter a shell after a failed build

In GitLab by [Gitlab user @abderrahimk] on Feb 15, 2020, 14:16
changed title from {-c-}annot enter a shell after a failed build to {+C+}annot enter a shell after a failed build

In GitLab by [Gitlab user @abderrahimk] on Feb 15, 2020, 14:16
changed the description

In GitLab by [Gitlab user @abderrahimk] on Feb 15, 2020, 14:16
changed the description

In GitLab by [Gitlab user @abderrahimk] on Jul 4, 2020, 18:59
So there are actually two bugs:

Buildstream assumes the "build tree" is in %{build-root}, which is only true for BuildElement https://gitlab.com/BuildStream/buildstream/-/blob/5daa22ae6f60568849254274fdbf57bde5b919d8/src/buildstream/element.py#L1651
Buildstream silently fails if it can't find %{build-root}, so this goes unnoticed until the buildtree is actually needed https://gitlab.com/BuildStream/buildstream/-/blob/5daa22ae6f60568849254274fdbf57bde5b919d8/src/buildstream/element.py#L1656

BuildElement marks both %{build-root} and %{install-root}, and sets the output directory to %{install-root}. What should be done to be generic enough is to collect the directories that the plugin has marked using Sandbox.mark_directory() with the exception of the output directory (set using Sandbox.set_output_directory(), which is the artifact directory).
Since buildstream now tries to reconstruct the sandbox of the failed build rather than persisting it, this is a regression compared to 1.x.

In GitLab by [Gitlab user @abderrahimk] on Jul 4, 2020, 18:59
So there are actually two bugs:

Buildstream assumes the "build tree" is in %{build-root}, which is only true for BuildElement https://gitlab.com/BuildStream/buildstream/-/blob/5daa22ae6f60568849254274fdbf57bde5b919d8/src/buildstream/element.py#L1651
Buildstream silently fails if it can't find %{build-root}, so this goes unnoticed until the buildtree is actually needed https://gitlab.com/BuildStream/buildstream/-/blob/5daa22ae6f60568849254274fdbf57bde5b919d8/src/buildstream/element.py#L1656

BuildElement marks both %{build-root} and %{install-root}, and sets the output directory to %{install-root}. What should be done to be generic enough is to collect the directories that the plugin has marked using Sandbox.mark_directory() with the exception of the output directory (set using Sandbox.set_output_directory(), which is the artifact directory).
Since buildstream now tries to reconstruct the sandbox of the failed build rather than persisting it, this is a regression compared to 1.x.

In GitLab by [Gitlab user @abderrahimk] on Jul 4, 2020, 19:00
changed title from Ca{-nnot enter a shell after a failed build-} to Ca{+ching buildtrees only works with BuildElement (cannot enter a shell after a failed build)+}

In GitLab by [Gitlab user @abderrahimk] on Jul 4, 2020, 19:00
changed title from Ca{-nnot enter a shell after a failed build-} to Ca{+ching buildtrees only works with BuildElement (cannot enter a shell after a failed build)+}


External Contributions

I am looking for a full CoseSign1 implementation in Rust, that supports all the signing algorithms and headers listed in the IANA COSE registry. Are you open to getting external contributions to complete your CoseSign1 implementation? If so, can I contact you offline (email, Zoom call, etc.) to discuss this further?

Not at the moment since this project is related to my thesis, but as I have the same interest in the CoseSign1 specifically, I will start developing the missing CoseSign algorithms (EdDSA with Ed448 and RSA-PSS) and headers (x5*, x5*-sender) soon.

Ed448, RSA-PSS and x5* added in new update.


Flag Analytics graph should show zero traffic data points

It seems that the graph does not show time points with a zero count - it should, and the graph should always show the full time period selected.

This is done, however I can see an API issue whereby the last date is returned twice with 2 different values

Is it always just the last date?

It looks like an API change has been made to make the period work which meant that the FE was showing just 1 day, I've adjusted the frontend so that it requests 28 days as before however it looks like there's now a max of 12 days.

Hmm can't replicate this in prod anymore - will close


Add MIT License in toplevel repository

Hey, I've been using your ansible role for quite some time and I'm very happy how it works. I noticed that it is mentioned to be MIT licensed in the galaxy meta file. Would you mind making this more obvious and add the MIT license in the toplevel of the git repo so GitHub shows this more prominently? This is the only thing keeping me from recommending your role wholeheartedly. Thanks!

Hello, sure, just added LICENSE file, thanks for advise!


x/build/app/appengine: local development server panics on pageload

Issue
Running dev_appserver.py locally causes the following issue:
$ dev_appserver.py  app.yaml
...
panic: open app/appengine/perf_graph.html: no such file or directory

goroutine 1 [running]:
html/template.Must(0x0, 0xbbcd80, 0xc0002ce060, 0x1)
	/home/rakoczy/.goenv/versions/1.13rc1/src/html/template/template.go:372 +0x54
main.init()
	/home/rakoczy/development/build/app/appengine/perf_graph.go:240 +0x6c4
panic: open app/appengine/perf_graph.html: no such file or directory

Cause
This was caused by a remediation for go111 module mode in app engine, which calculates repository paths on upload based on the location of the go.mod file in the repository.
See: https://github.com/golang/build/commit/c07384488cb9a1c2b60ef47769ede5221533ff37
Suggested fix
To workaround dev_appserver.py's differences with App Engine, we could detect if the file exists in app/appengine before falling back to a local file (for example, app/appengine/perf_graph.html and perf_graph.html as the fallback).
Ideally this is temporary until either we can easily create a nested module with AppEngine (see alternative 1 below), or until a fix for dev_appserver.py is completed.
Alternatives

Create a nested module in app/appengine/go.mod
This would involve all the complications of migrating to a nested module: https://github.com/golang/go/issues/27056#issuecomment-455566205.
I've explored this a bit. The biggest issue is that AppEngine doesn't currently respect rewrite rules at upload time, which means that adding the local dependency for golang.org/x/build requires a potentially non-buildable commit to be merged, followed by a correction commit.

Move app/appengine/app.yaml to the root of x/build, and supply a main directive that points to app/appengine.
Making this concession seems at odds with the organization of the x/build repo.

Move to Cloud Run and Docker.

See #34107, which discusses deployment time issues with GOPATH vs module mode.
/cc @dmitshur @andybons

Thanks for describing the issue in detail.
The suggested fix sounds very reasonable to me. It seems better compared to alternatives 1 and 2. Alternative 3 is also a good one
I want to offer another alternative for us to consider (either as a solution now, or a long term direction to move in):
Another way to resolve this problem is to move away from using dev_appserver.py for local development, and instead build and run the Go program normally (e.g., go run golang.org/x/build/app/appengine).
In the "Migrating your App Engine app from Go 1.9 to Go 1.11" document, one of the described changes is that it's possible to write an App Engine app that's more like non-App Engine apps, meaning it has a package main with func main() in it. (See https://cloud.google.com/appengine/docs/standard/go111/go-differences#package-main.) It just needs to os.Getenv("PORT") and use that for its HTTP server, but otherwise it can behave like a normal Go command.
We already do this for some projects that need to be deployed to App Engine but also run locally. For example, the tour. See CL 165537 for an example.
Some disadvantages of this approach:

it might be more work compared to the suggested fix
I'm not very familiar with the benefits that being able to use dev_appserver.py offer, but if there are some, we'd lose them

It's worth considering our longer-term goal of merging the build dashboard code into cmd/coordinator, as that work would make this problem obsolete.

Great point, I forgot about that option! I think that moving off dev_appserver.py using that guide could be the best fix, and I'll look into it first. If it's too much work, it might be worth doing the longer-term goal of merging the build dashboard into cmd/coordinator.

To workaround dev_appserver.py's differences with App Engine, we could detect if the file exists in app/appengine before falling back to a local file (for example, app/appengine/perf_graph.html and perf_graph.html as the fallback).

It seems CL 194643 implements this, just with an inverse order of the lookups. /cc @bradfitz

I believe this issue is resolved, see https://github.com/golang/go/issues/34116#issuecomment-530617975. @toothrot Do you concur?

I think we can consider this resolved. I'll file a new bug to move off dev_appserver.py, so we can run this as a normal go server.