Allow copying build logs easily

The terminal can be fickle to copy build logs from. We should allow people to download them.

Something I noticed:
If I do not uncollapse the log viewer right after loading the URL (and instead let Binder do its thing and uncollapse the viewer itself at the end), neither Firefox nor Chromium let me select any text in it.
Firefox does not let me copy the text at all.
Chromium requires me to right-click -> "Copy". Neither Ctrl-C nor Linux' usual select-to-copy copying work.

Arg - thanks for your input @kannes!! We agree this is a pretty frustrating piece of the Binder UX right now :-/  if we can store the text in an element somewhere on the page, it should be pretty easy to use clipboard.js to give a "copy" button, but we need to make sure not to accidentally crash something in the process

Done in https://github.com/jupyterhub/binderhub/pull/1335

Thank you!


new US contributor

Can someone add a new contributor to the US version of the survey?  Specifically, @sandragesing

I added her


Relay toggles on and off when connected to Arduino IDE

I was able to get this sketch working with a WeMos D1 Mini and the first relay of a 4-relay board. The problem I'm having is when I'm connected to and powered by my computer via USB. It's strange, when I plug in the WeMos, it connects to WiFi and behaves normally. Then I open up the Arduino IDE (version 1.6.8) and SOMETHING (not me) starts toggling the relay once per second.
So I thought I'd open up the serial monitor to see if any messages are being transmitted. Surprise! When I open up the serial monitor, it stops toggling and behaves normally. The mysterious toggling behavior correctly interpreted and displayed on my phone by the Blynk app. What is up?

now this is weird... i can only think that some default variable/status/ something is not right
opening the serial monitor causes a reset so it means it must work after a reset, which is weird
i ll try to test on a wemos at some point, although, i ve only tried on the sonoff with the serial monitor connected ... might need to test that as well

Issue resolved. I was using the Wemos pin "D3" for the button, but that corresponds to GPIO pin 0, which is used for entering programming mode. When the Arduino IDE is running and the WeMos D1 mini board is selected, and without the serial monitor open, it must be causing that pin to toggle about once a second so it's always ready to enter programming mode. So I switched to pin "D8", but that was a bad idea. The way the code is written, the button pin needs to be tied to 3.3v through a ~10k resistor and grounded to trigger the timer to start detecting if it's a long, medium or short press. If WeMos pin D8 (corresponding to GPIO 15) is high when the unit is reset, then it will not run the stored program or enter programming mode. Finally, I assigned pin D5 for button duties. This corresponds to GPIO 14 which doesn't have any special requirements when restarting.
Note: OTA programming is working! But when I start the Termite terminal program to view serial monitor messages, it resets the ESP. No big deal, but having GPIO 15 (D8) tied high had me confused why the program wouldn't run and it would not enter programming mode.


depth argument of Cell.get_labels() is confusing

I'm referring to the following code segment.
    def get_labels(self, depth=None):
        labels = libCopy.deepcopy(self.labels)
        if depth is None or depth >= 0:
            for element in self.elements:
                if isinstance(element, CellReference):
                    labels.extend(
                        element.get_labels(None if depth is None else depth -
                                           1))
                elif isinstance(element, CellArray):
                    labels.extend(
                        element.get_labels(None if depth is None else depth -
                                           1))
        return labels

Passing in depth=0, one gets labels from the top-level cell, plus labels from the first-level child elements.
In order to get labels from the top-level cell only, one has to pass in depth=-1.
This behavior was confusing to my colleagues. After reading the code, I think it's better to change one line:
        if depth is None or depth > 0:

@hashhsah Thanks for the alert. I've fixed it in the latest master.


Fix board indices

Fixes #13

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: pringon:x: BitBaitYou have signed the CLA already but the status is still pending? Let us recheck it.

I'm going to resolve the conflicts so that I can review.

Did I miss any suggested changes?

Did I miss any suggested changes?

There's a couple of unresolved conversations that I've now commented on. Also remember to merge master into this branch.

Is there anything else I need to do on this one?

This is too complex to rebase and merge unfortunately.


Line height metrics differ between local & Travis

While working on #592 I've run into a baffling difference in font metric calculations between systems. Perhaps @simoncozens (or @khaledhosny or @behdad) you have ideas?
I've gotten everything coming out the same across platforms and most library versions, but for some reason when we compute line height metrics for Libertinus Serif or Noto Sans CJK we are coming up with different answers on some systems (Arch Linux and OSX give the same answer) vs. others (Travis running Ubuntu Xenial gives a different answer). I've tried with multiple versions of Harfbuzz on Travis (and currently using the latest 2.5.1 matching my Arch Linux) and the line height metrics are still different.

Same font files (Libertinus 6.9, etc.)
Same Harfbuzz version (and the Harfbuzz's metrics are used, not Freetype's)
Both local and Travis using fontconfig to find fonts (not the same version, but all this does is pass the font file off to Harfbuzz right?)
Of course the same SILE code & shaper

What else is there that could affect our line height calculations only for these fonts and not for all others?
Note only lines in which the tallest thing in the line is these fonts matter. The fonts can be included in the document and text rendered, as long as something on the line is larger than these fonts.
By example this works:
\font[family=Gentium Plus,size=15pt]{Gentium \font[family=Libertinus Serif,size=12pt]{Libertinus}!}

But this fails:
\font[family=Gentium Plus,size=12pt]{Gentium \font[family=Libertinus Serif,size=15pt]{Libertinus}!}

...And by fails I man the line metrics are ~0.2pt different in the version where Libertinus sets the pace.

OK, so one commonality is that Libertinus Serif and Noto Sans CJK are CFF fonts.
I'm trying to replicate a Travis environment locally with docker. Meanwhile, let's think how to reduce this to its simplest form:

The problem is differing leading calculations
Leading is calculated on baselineskip (which is constant) and the depth of the previous line.
0.2pt is about the right order of magnitude for a change in depth.
How do we compute the depth of a glyph?

      double height = extents.y_bearing * point_size / upem;
      double tHeight = extents.height * point_size / upem;
      lua_pushnumber(L, -tHeight - height);

Here's something run from the SILE repl which may help diagnose:
SILE.shapers.harfbuzz:shapeToken("a",SILE.font.loadDefaults({filename=".fonts/LibertinusSerif-Regular.otf"}))

Running ./sile <<< 'SILE.shapers.harfbuzz:shapeToken("a",SILE.font.loadDefaults({filename=".fonts/LibertinusSerif-Regular.otf"}))' locally I get:
{1={depth=0.1,gid=66,glyphAdvance=4.57,height=4.39,index=0,name=a,text=a,width=4.57}}

Travis is producing:
{1={depth=0.1,gid=66,glyphAdvance=4.57,height=4.56,index=0,name=a,text=a,width=4.57}}

Looking very harfbuzzy, then. Here's the MWE as a C file:
/* gcc -o test-hb test-hb.c `pkg-config --libs --cflags harfbuzz` */
#include <stdio.h>
#include <assert.h>
#define MAXBUFLEN 1000000
#include <hb.h>
#include <hb-ot.h>

int main() {
  char font_s[MAXBUFLEN + 1];
  FILE *fp = fopen(".fonts/LibertinusSerif-Regular.otf", "r");
  assert(fp);
  size_t font_l = fread(font_s, sizeof(char), MAXBUFLEN, fp);
  assert(!ferror(fp));
  font_s[font_l++] = '\0';
  hb_blob_t* blob = hb_blob_create(font_s, font_l, HB_MEMORY_MODE_WRITABLE, (void*)font_s, NULL);
  hb_face_t* hbFace = hb_face_create(blob, 0);
  hb_font_t* hbFont = hb_font_create (hbFace);
  unsigned int upem = hb_face_get_upem(hbFace);
  hb_font_set_scale(hbFont, upem, upem);
  hb_glyph_extents_t extents = {0,0,0,0};
  hb_font_get_glyph_extents(hbFont, 0x61, &extents);
  printf("Height = %i\n", extents.y_bearing);
  return 0;
}

I get Height = 439 on this Mac.

Running this across different HB versions compiled on Travis I get the following:

Harfbuzz 2.5.1
{1={depth=0.1,gid=66,glyphAdvance=4.57,height=4.56,index=0,name=a,text=a,width=4.57}}

Harfbuzz 1.4.2
{1={depth=0.1,gid=66,glyphAdvance=4.57,height=4.56,index=0,name=a,text=a,width=4.57}}

Harfbuzz 1.1.3
{1={depth=0.1,gid=66,glyphAdvance=4.57,height=4.56,index=0,name=a,text=a,width=4.57}}

Hey at least it's consistent.

I get Height = 439 on my Arch Linux too.

@ebraminio? Something funky going on with CFF get_extents.

Just for grins, I'll throw our old mystery issue with Graphite enabled metrics in here too in the off chance it might actually be related, see #386.

I was just thinking about Graphite...

I now have a running Travis image as a docker container.  OK, here we go:
Travis:
# /usr/local/bin/hb-shape  --version
hb-shape (HarfBuzz) 2.5.1
Available shapers: ot,fallback
# md5sum .fonts/LibertinusSerif-Regular.otf
3409bacd6d3a6a75087494c4b8e26b6d  .fonts/LibertinusSerif-Regular.otf

#  hb-shape --shapers=ot --show-extents .fonts/LibertinusSerif-Regular.otf 'a'
[a=0+457<27,456,437,-466>]

Mac:
$ hb-shape --version
hb-shape (HarfBuzz) 2.5.1
Available shapers: graphite2,ot,coretext,fallback
$ md5sum .fonts/LibertinusSerif-Regular.otf
3409bacd6d3a6a75087494c4b8e26b6d  .fonts/LibertinusSerif-Regular.otf

$ hb-shape --shapers=ot --show-extents .fonts/LibertinusSerif-Regular.otf 'a'
[a=0+457<36,439,419,-449>]

After rebuilding on the Mac without graphite:
$ hb-shape --version
hb-shape (HarfBuzz) 1.9.0
Available shapers: ot,fallback
$ hb-shape --shapers=ot --show-extents .fonts/LibertinusSerif-Regular.otf 'a'
[a=0+457<36,439,419,-449>]

What the ...?

I'll post in Harfbuzz repo.

@ebraminio? Something funky going on with CFF get_extents.

get_extents can use freetype callbacks, Travis CI is known to have old versions of freetype and they versions can be different so test with --font-funcs ot and see what happens.

    /* Harfbuzz's support for OT fonts is great, but
       there's currently no support for CFF fonts, so
       downgrade to Freetype for those. */
    if (strncmp(font_s, "OTTO", 4) == 0 || strncmp(font_s, "ttcf", 4) == 0) {
      hb_ft_font_set_funcs(hbFont);
    } else {
      hb_ot_font_set_funcs(hbFont);
    }

This comment is no longer true.

This comment is no longer true.

Yep

Just curious to know, is it possible for sile to go without CoreText and Freetype and ship [an embedded version or not] of harfbuzz? Also cool if you can use hb_blob_create_from_file for font file reading!


aliases for serial tx/rx

@TheZoq2 What do you think about this change? I don't have any particular opinion here.

I don't really have any strong opinion either.
If i'm reading this right, this would be a breaking change, right? Then maybe it makes sense to wait with merging it until we have a few more breaking changes. That way we have the option of doing a minor release or two in between

This change is not breaking. Only shortcuts for type names to simplify their use in statics.

Oh, I missread it then, in that case, i'd be happy to merge it if you add a changelog entry :)


Dotnet nuget publish is failing

What happened?
Nuget publish is failing due to unsupported image formats:
error: Response status code does not indicate success: 400 (Unsupported icon image format. Only PNG and JPG images are supported.).

https://github.com/pulumi/pulumi-command/actions/runs/4491757899/jobs/8273965607
Expected Behavior
It succeeds
Steps to reproduce
Look at "Publish SDKs" here: https://github.com/pulumi/pulumi-command/actions/runs/4491757899/jobs/8273965607
Output of pulumi about
.
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

Looking at the PNG, it's an SVG file: https://github.com/pulumi/pulumi-command/blob/master/sdk/dotnet/logo.png
Which is not a PNG, so i think Nuget is correct here.

@zlepper Thank you for discovering and fixing this!


Ability to change {{ }} to other symbols

in order to work with blade templating (laravel), i had to change angular start-end symbols
$interpolateProvider.startSymbol('<%');
$interpolateProvider.endSymbol('%>');
Could be possible to enable this changes in paramters in angularcomplete-alt ?

I would just provide my own custom template at this point :-)

I thought it was difficult but turned out it wasn't.
Now it's automatic.

Blade permit to escape the {{ }} with a @{{ }} so it's better doing so and removing
$interpolateProvider.startSymbol('<%');
$interpolateProvider.endSymbol('%>');

If that's the case, you still need to provide your own template.
Using $interpolateProvider is the proper angular way to specify your own interpolate characters.
There was an issue #168 so I had to do this anyway.


No token type

I don't know if this is intended or not, but when I create a new token like documentation says:
String token = JWT.create()
    .withIssuer("auth0")
    .sign(Algorithm.HMAC256("secret"));

There is no type field on header, I haven't tried other algorithms but I think it should be the same.
In the example of jwt.io there is a 'typ' field on headers.
Some places say you should check type to prevent tokens that you don't expect, I know this doesn't improve security, it's just a filter to discard malicious tokens. Anyway I'm curious about type tag.

Thanks, we've added the token type claim when the token is signed.
PR: https://github.com/auth0/java-jwt/pull/138


Update vue.config.js

changed django to flask

Codecov Report

Merging #28 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #28   +/-   ##
=======================================
  Coverage   98.03%   98.03%
=======================================
  Files           5        5
  Lines          51       51
=======================================
  Hits           50       50
  Misses          1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 654c2ba...b0cfed4. Read the comment docs.


,.

Yes, it does. There are examples at https://github.com/mozilla/pdf.js/tree/master/examples/node.

Sorry I mis-pressed enter key. So there is no content in the issue.


Start upgrade to hyper 1.0 RC1

Start on the upgrade to hyper 1.0 RC1 in a separate branch of smithy-rs.

I spent some time working on this this week. I was able to get aws-smithy-http compiling with most tests passing, and some of aws-smithy-checksums (ended up stubbing out the checksum calculating body to move forward). My work is available on the https://github.com/awslabs/smithy-rs/tree/hyper-1x-experimentation-dec-7-2022 branch. Findings below.
Further work that is unblocked right now:

[ ] Fix failing tests in aws-smithy-http after upgrade
[ ] Reimplement checksum calculating body in aws-smithy-checksums to support new Body trait

Prerequisites before continuing:

[ ] Need hyper-util to release with a HttpConnector trait and connection pool
[ ] Need an upgraded TLS library (hyper-tls or hyper-rustls) to actually test the SDK. We can do this work ourselves if the previous prerequisite is completed.

Work that can be done after prerequisites:

[ ] Upgrade or wait for hyper-rustls to support hyper 1.x
[ ] Upgrade or wait for hyper-tls to support hyper 1.x
[ ] Fix dvr in aws-smithy-client (transition from hyper::body::channel() to http_body_util::StreamBody)
[ ] Continue investigation

Note: On the Body impls, it should be safe to assume trailers come after data frames since hyper will error out otherwise.

Findings so far:

Depending on what hyper-util ends up supporting, we may need to fully implement the HTTP versioning RFC to avoid needing to worry about ALPN (or it may be cheaper to do so).


Fix playground code action

Summary
This PR fixes the playground code action to start working again. It seems that
the field name was changed from edit to textEdit in some version.
Resources:

https://microsoft.github.io/monaco-editor/docs.html#interfaces/languages.IWorkspaceTextEdit.html
https://stackoverflow.com/a/71742764

Test Plan
Tested it out running locally.

Current dependencies on/for this PR:

main

PR #7824   👈

This comment was auto-generated by Graphite.


Add pending status to Rock Financial model and pull it in via NMI

@benwiley @dcs619 I'm thinking a DefinedType for transaction status might be a good way to go about this? Then add Status to the transaction model and set the DefinedType value for the status of pending, complete, failed. It would be nice, although not sure if possible via NMI (I'll check) to pull the reason for failure too. I'm planning on chatting with Spark tomorrow about this.

It doesn't look like NMI gives a reason for failed charges but I'm still looking into it

We also should sync the status and show a user a gift is pending on the UI cc @samclaridge

@jbaxleyiii @samclaridge


Making development compose file more similar to production

This is an improved version of #493.

Makes it possible to use the Docker Compose file in development again.
Documents how to launch the services locally using Docker Compose.

I thought the text would overwhelm the existing content in the README, so I made these changes:

Left a comment in the docker-compose.yml file to direct viewers to the file mentioned in the next point.
Moved most of the content into a new Markdown file dedicated to documenting the use of Docker Compose for the website (running-with-docker-compose.md).
Put a small blurb about it in the README and linked to the new Markdown file.


Remove version: Wesnoth.BattleForWesnoth version 1.15.11

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://sourceforge.net/projects/wesnoth/files/wesnoth/wesnoth-1.15.11/wesnoth-1.15.11-win64.exe/download
Status Code: 200
(Automated message - build 897)

Close with reason: Package still available.;


add  "postbox-notification"

By the way, what happens if there is no translation for some entry? Isso uses English? For example, there is no Russian translation for "atom-feed", but I think it is OK to display "Atom feed" for Russian users.

I must admit that I don't know this part of the project yet. So I'm not able to answer…
And to be honest if I ever go into the JS part of it I rewrite it all in some more modern way.


Turkish Character Problem

When I tried to upload a file which is named as "çiçek.jpg", plugin doesnot recognize the "ç" and exchange the character "ç" with correponding charset code (File name changed by the plugin). So file is uploaded but it could not find the file again, because MySQL can handle this and it reserve the orginal file name as "çiçek". Any way to say your plugin to use charset like utf-8? Thanks!..

You could use the file rename callback method to solve this and rename the file as you see fit. The plugin doesn't have a character set other than what you configure in your application.

But my application charset was set to UTF-8. Thus, it should work properly. I will apply your solution. Thanks...

Well with the filename it'll be down to the hosting machines system rather than your application. So it could be that your hosting doesn't support the character.
As I say, I'd personally use the rename callback method and use the PHP Intl extension to transliterate the characters so they'll save properly.

Problem solved. Thanks...


Automatically generate type-safe accessors for contributed extensions

Apply plugins block against empty project and see which extensions are added.
Then use code generator and build script compiler as in the metadata file driven solution.

pros

it is automatic

cons

won't take imperative plugins into account

I can't reproduce what I see in the screencast. Is it on master yet?

It is.
You have to be running the latest Kotlin plugin from the EAP channel (1.1.0-rc-91-*) and reimport samples/hello-world.

Thanks, @bamboo. It's working for me with Kotlin plugin 1.1.0-release-IJ2016.3-1.
I ended up cloning a fresh copy of the project to make it work. "Refresh all Gradle projects" and importing project again didn't help.


⚠ ClamAV has degraded performance

In db07d6b, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 443 ms

Resolved: ClamAV performance has improved in 24e99db after 5 minutes.


Bump Sphinx to 1.3.5

Fixes #1966
This just requires testing at this point.

Actually, this should probably target 1.3.6, which is due out soon:
https://groups.google.com/forum/#!topic/sphinx-dev/ljzyL8KRCss
Not sure on conda release timeframe.

We should merge 1.3.5 support first, and we can upgrade when it's released.


Add eospac variant to spackage

PR Summary
Make eospac a variant in singularity-eos spackage. (Eospac was already listed as a dependency)
Spack looks for eospac in the spec to determine if eospac should be pulled in:           self.define("SINGULARITY_USE_EOSPAC", "^eospac" in self.spec)
PR Checklist
Spack env builds on CTS1_intel.

[ ] Adds a test for any bugs fixed. Adds tests for new features.
[ ] Format your changes by using the make format command after configuring with cmake.
[ ] Document any new features, update documentation for changes made.

@dholladay00 @annapiegra @Yurlungur ,
I would prefer that the use-case be made clear in the interface. Here, the variant would be (as example, could be different)
variant("eospac_wrapper", default=True, description="build the EOSPAC wrapper")
More broadly (and this is probably a separate PR), the CMake has a lot of overlapping inter-dependencies that make it difficult to sort out a clean dependency trees. For example, at CMakeLists.txt:333
if(SINGULARITY_BUILD_SESAME2SPINER)
  if(SINGULARITY_USE_EOSPAC)
    if(SINGULARITY_USE_HDF5)

Now, the spack spec singularity-eos~eospac build_extra=sesame is valid, but due to the logic of the build chain sesame2spiner will not be built.
Further, in nearly all cases where SINGULARITY_USE_EOSPAC is checked to alter some extra compiler work, it is used in conjunction with SINGULARITY_USE_HDF5, which implies that if we depend on EOSPAC, we depend on HDF5 for any meaningful use-case. However, it is explicitly allowed (though warned) to enable EOSPAC without linking to HDF5.
This was my original dilemma is writing the package.py files; What would be ideal (from my point-of-view) is to enforce a cleaner dependency structure. For example, if I wanted to build ``singularity-eos build_extra=sesame, this would automatically give EOSPACandHDF5` as dependencies:
# ensure hdf5 and eospac are installed together
depends_on('hdf5', when='+eospac_wrapper')
# do not allow sesame2spiner to be built if there is no eospac dependency
conflicts('~eospac_wrapper', when='build_extra=sesame')

The issue then is should the dependency range implied by the current CMake logic what we want, or is there a more "correct" dependency graph.

I think it is possible to want eospac but not hdf5 if, for example, the eospac backend is used without spiner.

Updated package.py to use eospac variant @annapiegra @Yurlungur

Does the cmake options file that's generated need to be updated or is that automagically done @mauneyc-LANL ?

Does the cmake options file that's generated need to be updated or is that automagically done @mauneyc-LANL ?

we have
 self.define("SINGULARITY_USE_EOSPAC", "^eospac" in self.spec)

as the cmake option, afaik that's the only one

@annapiegra is this still needed? Or has this been resolved?

@annapiegra is this still needed? Or has this been resolved?
Some version of this is needed. I think @danl and @mauneyc-LANL understand the dependencies better.

@mauneyc-LANL @annapiegra should I merge this? I see we have approvals

@mauneyc-LANL @annapiegra should I merge this? I see we have approvals

This looks fine to me, if @annapiegra is okay with it

Fine with me.

Merging then


Content Suggestion: casebitcoin.com

Link to Content: https://casebitcoin.com/
Briefly Describe Content: Key bitcoin data & News
Suggested Section: Network / Metrics

No obvious place for this to go. Combination of Data & News. Is there enough content to merit a News section on the Learning Page?


Segmentation fault: 11 in conda env

I am running a python 3.9 conda environment with the copy pasta code provided
the segfault occurs on the first line: model = load_model('base')

Look like it might a PyTorch issue. Have you tried loading it just from whisper?
import whisper
model = whisper.load_model('base')

I did, it didn't work with base whisper either. I decided to try further downgrading my python via conda venv and it worked, I used 3.8 down from 3.9.


Update zh-rTW strings.xml

Thanks, these changes will be verified within 5 hours before accepted.

Changes accepted into one single commit, see d580bfd60341bdf3ec789fd115e1157f1bc838c1


[RFC] Add logging to Toolpad

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Summary 💡
From https://github.com/mui/mui-toolpad/issues/592
Add logging to Toolpad.
What should we log?

Frontend application logs
Request tracing
REST datasource requests

What logging aggregation service should we use, if any?
Some popular ones are: Datadog, New Relic, Grafana Loki (more complicated to set up)
Feel free to suggest others.
Examples 🌈
No response
Motivation 🔦
No response

I'm wondering what exactly we would like to see at this moment? Maybe error logging would be enough for now until we know better?

I'm wondering what exactly we would like to see at this moment? Maybe error logging would be enough for now until we know better?

Maybe the logs Nextjs gives us out of the box are enough to start with. I think that would also include any errors if the application crashes for example.

I want to be able to get insight into the requests that are made from the backend to external hosts. To be able to gain insight into load and spot abuse.

What logging service should we use? Some popular ones are: Datadog, New Relic, Grafana Loki (more complicated to set up)
Feel free to suggest others.

https://github.com/winstonjs/winston

Ideally

we log to stdout, the host environment is responsible for collecting logs. This is important for self-hosters.
each log line is a json message, in a standard format containing timestamp, message, severity and metadata. I'm familiar with bunyan and pino.
logs are searchable, filterable, can be aggregated and dashboards can be built based on metadata. I'm familiar with elasticsearch/kibana for this purpose.

regarding 1. it seems that render.com has more or less an all-or-nothing approach with regards to log shipping: https://render.com/docs/log-streams. This may be a quick and easy solution for us. I'm not quite sure about predicting the cost. alternatively we'll have to ship logs from within the application but I don't want to support that option towards self-hosters.
regarding 2. maybe not top priority, but towards self hosters we may have to provide an option to format logs. For me it's important that we standardize on log messages (e.g. standard fields to attach errors, req, res objects with their own formatters,...), and be able to attach metadata.
regarding 3. I've seen powerful things being done with the ELK stack, it has a learning curve though. Alternatively we can look into any of the options render.com proposes

Thanks for the feedback - will switch to pino and it seems to handle overhead better - I agree with the rest but I think my initial implementation already follows most of these things, should be just a matter of fine tuning it.

To clarify, I like pino, but I'm fine with winston if the team prefers. I think the main thing I would change is to remove the log formatting from the app and instead have it output raw jsonline. Then for dev move the formatter to its own process by piping the start command into it. (And for prod we can do the same for now, and replace with an elasticsearch logshipper if we want to add that in the future)


Build failure on Jessie

The build script started failing recently with the following message:
mkfs.fat 3.0.27 (2014-11-12)
Loop device does not match a floppy size, using default hd params
Formatting root partition on /dev/loop0...
mke2fs 1.42.12 (29-Aug-2014)
Invalid filesystem option set: none,sparse_super,large_file,filetype,resize_inode,dir_index,ext_attr,has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
rootfs.mak:176: recipe for target 'Raspbian-jessie-rpix' failed
make[1]: *** [Raspbian-jessie-rpix] Error 1
Makefile:21: recipe for target 'build-rootfs' failed
make: *** [build-rootfs] Error 2

I am running Debian Jessie with all packages up to date.
I was able to fix this by manually editing rootfs/etc/mke2fs.conf and remove metadata_csum from the ext4 features line, then run createimg manually, but it would be great if we can fix this in the build script as well.
Looks like the mk2fs Jessie installs is too old to support this new file system option.

 I keep going back and forth.  I'm going to try and revisit how I do this to be more robust, thanks for letting me know it's back again.

Just started using this script. This thread says that updated packages have been applied to the Rasbian mirrors recently. Though I still get the issue.
https://www.raspberrypi.org/forums/viewtopic.php?t=186769&p=1178842

it's just the options I use when building the image, I'm still trying to find a more sane method to deal with this, I thought taking the defaults for the HOST OS and the defaults from the IMAGE, and crossing them would work but seems something is wrong in that logic that I'm trying to figure out.

@dndx can you run this and post the results for me please
echo $(cat /etc/mke2fs.conf | grep "base_features = " | cut -d "=" -f2 | tr -d '[[:space:]]'),$(cat /etc/mke2fs.conf | grep -A3 "ext4 =" | grep "features = " | cut -d "=" -f2 | tr -d '[[:space:]]')

Believe this is fixed now


Default value for attenuation in attenuated-MIP rendering should be lower

🐛 Bug
I'm not sure if it's my data or a general feature, but I have always found that the attenuation value for 3D rendering is way too high for most volumes I look at. I invariably end up reducing it to just above the minimum value of the slider. So we should probably revisit that default value.

Interesting, might be good to think why that is/ if more normalization is needed or if the value is just too high!

The fix worked beautifully!  As you know, I used an attenuation value of .04 for this dataset.  However, I'm going to test a couple of different datasets out with the attenuated MIP setting and I'll get back to you as soon as possible on the range of values I find useful.  Thanks again, I really appreciate your work and I think you are building the foundation of something potentially transformative.
Best,
Kevin

a while back we had (on slack) talked about, and rejected using a nonlinear $cmap function in the shader.... pasting that code here again in case we want to play with it:
ATTENUATED_MIP_SNIPPETS = dict(
    before_loop="""
        float maxval = -99999.0; // The maximum encountered value
        float minval = 99999.0; // The minimum encountered value
        float sumval = 0.0; // The sum of the encountered values
        float scaled = 0.0; // The scaled value
        int maxi = 0;  // Where the maximum value was encountered
        vec3 maxloc = vec3(0.0);  // Location where the maximum value was encountered
        """,
    in_loop="""
        sumval = sumval + val;
        scaled = val * exp(-u_threshold * sumval / u_relative_step_size);
        if( scaled > maxval ) {
            maxval = scaled;
            maxi = iter;
            maxloc = loc;
        }
        if( val < minval ) {
            minval = val;
        }
        """,
    after_loop="""
        gl_FragColor = $cmap((maxval - minval) * (1 + pow(u_threshold, .5)));
        """,
)

Linking to this comment with great images, examples https://github.com/napari/napari/issues/1399#issuecomment-658866839
Seems like values there are 0.02, 0.06, 0.04.
How about we bring the slider range down to (0, 0.2) with a default of 0.1. I can do that now


Add git ignore for vscode

Description
Add git ignore for vscode
GitHub Issues
n/a
Checklist:

[x] I have performed a self-review of my own code to ensure there are no formatting, vetting, linting, or security issues
[ ] I have verified that new and existing unit tests pass locally with my changes
[ ] I have not allowed coverage numbers to degenerate
[ ] I have maintained at least 90% code coverage
[ ] I have inspected the Grafana dashboards to verify the data is displayed properly
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have maintained backward compatibility

How Has This Been Tested?
n/a
Manual inspection of the GUI
n/a

run e2e test

e2e pass

run e2e test

run e2e test

run e2e test

run e2e test

run e2e test

run e2e test


Protect Cesium from crashing when the camera goes through terrain

This fixes #6783, at least for the exact error shown in the first stack trace posted on that issue.

Thanks for the pull request @emackey!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

:grey_question: Unit tests were not updated.

Make sure you've updated tests to reflect your changes, added tests for any new code, and ran the code coverage tool.

Reviewers, don't forget to make sure that:

[ ] Cesium Viewer works.
[ ] Works in 2D/CV.
[ ] Works (or fails gracefully) in IE11.

I can't seem to reproduce the crash in master right now, but the fix looks good.


suppress denial logspam

These denials are not causing app bugs but the underlying code needs
to be fixed. In the meantime, remove them from the logs.
Bug: 63631799
Test: Build and flash, verify denials are gone.
Change-Id: I6ddc91aa792b771e2c4190b211588347cc4119ea

@AdFad666 @AdrianDC

I don't really like this.
As it widely silences proc: and device: for example, it could miss real denials that we would want to care about and to fix, except they wouldn't show unless this is reverted / removed on purpose.
I prefer using dontaudit only on very specific denials that we know to be irrelevant, example :
https://github.com/LineageOS/android_device_sony_huashan/blob/lineage-15.0/sepolicy/dontaudit.te
https://github.com/LineageOS/android_device_sony_huashan/commit/bc4d7444082eb16e48a8f21eebbbce4866c4afdb#diff-d54a18733b359d66154d1dc5fa179049

We don't have them

@adrianDC we have some priv_app denials same which wahoo have dontaudit due to same spam reason

I believe you have, otherwise I hope you wouldn't be pushing this commit.
I'm just worried you're not going to hide too generic nodes / denials with this, which would later need to be fixed but will go unnoticed because they'll be ignored.
Also we don't know what specific denials this concerns, please comment with some related denial logs here.

Adrian the denials are numerous for example https://github.com/sonyxperiadev/device-sony-sepolicy/pull/269/commits/8768a85140e270dc194213316f27c6a08049ca5e
The priv_app one

You're showing me that we did resolve some priv_app denials in the past, and at the same time, you want to "donotaudit" some of these denials.
donotaudit doesn't allow at all, it simply hides denials we know 100% to be invalid or irrelevant, hence I'll stay on what I said in my previous comment.


reflect: MakeInterface

Feature request for MakeInterface, to go along with the newly-arrived MakeFunc (Issue
1765).

Good goal would be gomock without source code generation.

@bradfitz You mention having to implement wrappers for optional interfaces of http.ResponseWriter, are there any examples in http standard library? I've come across this problem as you've described it: https://groups.google.com/d/topic/golang-nuts/HKY3mI7Q2jY/discussion

@crawshaw, any interest? :)

@bradfitz I don't have time for this in the 1.7 window.

I have been considering implementing this for 1.8. There are some details in the runtime's itab code to work through, but that aside I think the API needs a little more consideration. It's not really MakeInterface because it's not making an abstract interface type. Instead the function is creating a new concrete type with a method set.
When writing Go we can add a method set to any named type. So I lean towards a function for creating a new named type, with an optional method set:
func NamedOf(t Type, name string, methodNames []string, methods []func(args []Value) results []Value) Type

This could reuse the reflect.Method struct for what looks like a simpler signature, but it will add a few new states to the Method struct (for example, will the Type field be used when passed to NamedOf?):
func NamedOf(t Type, name string, methods []Method) Type

@bradfitz does this work for your http.ResponseWriter case?

I'm fine with reflect.Named, but it's not the same as reflect.MakeInterface.  The meaning of reflect.MakeInterface (or reflect.InterfaceOf) would be to create a new interface type that could then be used with reflect.Value.Convert to produce a new interface value (and you could use Method(0) to get the first method, etc.).  I don't know why such a thing would be useful, but it is meaningful.

I agree that MakeInterface is different (and valid in its own right), and if someone can think of a good use for it I'll give it a go too. Brad's earlier example looked like it needed NamedOf instead, so I co-opted this issue. I'll make another.

The ability to create a new types with a method sets would greatly reduce the boilerplate in many use cases I have. I find myself forced to implement various interfaces(i.e.  MarshalJSON/UnmarshalJSON,  datastore.PropertyLoadSaver) when I need to unmarshal data into types with overlapping fields even if the process/body of the methods is basically same except the types involved. See a stub below[0]. This could be generalised [1] if there was a way to create types with method sets.
[1]
// NewQueryDecoder creates a virtual struct using StructOf and implements PropertyLoadSaver
dq := NewQueryDecoder(reflect.TypeOf(T1{}), reflect.TypeOf(T2{}), reflect.TypeOf(T3{}))
// Execute a query
// dq.Query wraps the datastore query, allocates a new value using the virtual struct
//  and returns the value matching one of the types from NewQueryDecoder as interface{}
result, err := dq.Query("filter1=", true, "filter2=", false)
// type assert
switch result.(type){
case T1:
case T2:
case T3
}

type T1 struct{
  F1 string
  FT1 bool
}

type T2 struct{
  F1 string
  FT2 bool
}

type T2 struct{
  F1 string
  FT3 bool
}

[0]
type TOneOf struct{
	Type string `datastore:"@type"`
	*T1
	*T2
	*T3
}

func (x *TOneOf) Load(ps []datastore.Property) error {
	For k := range ps{
		if ps[k].Name != "@type"{
			continue
		}
		x.Type = ps[k].Value.(string)
		break
	}
        st := reflect.TypeOf(x)
	switch x.Type{
	case RefReflect(st.Field(1)):
		val = reflect.New(st.Field(1))
		if err = datastore.LoadStruct(val.Interface(), ps); err != nil{
			return err
		}
		reflect.ValueOf(x).Field(1).Set(val.Elem())
		return nil
	case RefReflect(st.Field(2)):
		val = reflect.New(st.Field(2))
		if err = datastore.LoadStruct(val.Interface(), ps); err != nil{
			return err
		}
		reflect.ValueOf(x).Field(2).Set(val.Elem())
		return nil
	case RefReflect(st.Field(3)):
		val = reflect.New(st.Field(3))
		if err = datastore.LoadStruct(val.Interface(), ps); err != nil{
			return err
		}
		reflect.ValueOf(x).Field(3).Set(val.Elem())
		return nil
	}
	return errors.New("Invalid type: "+  x.Type)
}

// RefReflect returns the type path (importPath.Type)
// e.g. encoding/json.Encoder
func RefReflect(t reflect.Type)string{
  // stub
    return t.String()
}

@crawshaw, you wrote:

and if someone can think of a good use for it I'll give it a go too.

I found in https://go-review.googlesource.com/#/c/27321/3/src/sort/sort.go (prototype for #16721) that the the interface wrapper around my funcs eats a ton of my time. If a MakeInterface let me avoid that, I would love it. But that sort code couldn't depend on reflect anyway, so it probably wouldn't help me regardless. :-/

Oh, but it'd still help me if I decide to put this whole slice sorting helper in its own package.

You want to create methods, right? That's a job for #16522, which I'm having a hard time with. A function named MakeInterface would create a new interface type, like interface { F() Foo }. Which is much easier but maybe not useful?

Yeah, sorry, I want to create an interface value with methods I define, implementing some existing interface. This bug is about creating an interface type now I guess? But #16522 also says its signature returns a type.
Which is about creating a value?

Well once you have the type you can use reflect.New(t).Elem() to get the value.
(Maybe it was a mistake to move the original meaning of the bug, perhaps they should have swapped titles.)

Oh, sure, to get it with its zero concrete value. I guess I just don't care about its concrete value so I was ignoring that.
I just want to set the itab values, basically. But now I realize from https://github.com/golang/go/issues/16522#issuecomment-237012698 that it's not what I want, since my function pointers are go11func pointers instead. And they're actually closures, so I can't cheat and hand you just the code half. I'll follow along in #16522

Hi,
I've written a user experience report that references this ticket in a potential solution to what I've seen as a larger problem.  I hope it can either help prioritize this ticket, or give context to what I would want in an ideal solution.
https://medium.com/@cep21/interface-wrapping-method-erasure-c523b3549912

@cep21, great write-up!

@cep21
Hmm, that's an interesting take on it. I've hit the wrapping problem several times myself, but I hadn't really made the connection between that and MakeInterface.
My takeaway from it was that we should stop using "magic methods" and replace them with some kind of compile-time introspection (e.g. some kind of metaprogramming support as part of #15292). All the complaints people have about the complexity of compile-time metaprogramming seem to apply equally well (if not more) to this kind of run-time metaprogramming.

I'm having trouble seeing how this would solve the interface wrapping method erasure problem.
Would the signature be something like MakeInterface(methods []struct { Name string; Fn func(args []Value) (results []Value)) Value and the returned value would be an essentially anonymous type with the specified methods?

@jimmyfrasche I would expect something a bit more dynamic than that. By analogy with StructOf and MakeFunc, and interpreting MakeInterface as roughly an inverse of (reflect.Value).Method:
// InterfaceOf returns the interface type containing methods. The Func
// and Index fields are ignored and computed as they would be by the compiler.
func InterfaceOf(methods []Method) Type {
	…
}

// MakeInterface returns a new interface of the given Type that wraps methods.
// len(methods) must equal typ.NumMethod(), and the Value for methods[i] must
// have a type assignable to typ.Method(i).Type.
func MakeInterface(typ Type, methods []Value) Value {
	…
}

I was thinking len(computedMethods) would need to match the length of the underlying type.
Forwarding unexported methods makes for an interesting problem, though: reflect.InterfaceOf would presumably need to support unexported methods, but MakeInterface shouldn't allow the caller to inject them (other than by forwarding to an existing implementation).
I think that implies an extra constraint on MakeInterface:
// If typ.Method(i).PkgPath is not empty, methods[i] must be the result of a call
// to (Value).Method for a method with the same package-qualified name.

Then the forwarding code would be roughly:
func forwardMethods(wrapper, underlying error) error {
	type methodName struct{Name, PkgPath string}
	seen := make(map[methodName]bool)
	var (
		methods []reflect.Method
		impls []reflect.Value
	)
	for _, v := range []reflect.Value{reflect.ValueOf(wrapper), reflect.ValueOf(underlying)} {
		t := v.Type()
		for i := 0; i < t.NumMethod(); i++ {
			m := t.Method(i)
			name := methodName{m.Name, m.PkgPath}
			if seen[name] {
				continue
			}
			seen[name] = true
			methods = append(methods, m)
			impls = append(impls, v.Method(i))
		}
	}
	return reflect.MakeInterface(reflect.InterfaceOf(methods), impls).Interface().(error)
}

But for this API to be useful for wrapping methods, it would need to have the (somewhat odd) property that reflect.MakeInterface actually produces a value of a concrete (non-interface) type, against which type-assertions can be made.

@bcmills I think that's why I find this confusing. The name MakeInterface is kinda off.
MakeInterface, in your example above, is making a non-interface type which conforms to an interface (which seems redundant since it's given the Methods which contain all the necessary information). So what it's really doing is making a named type (or whatever they're called now) without a name but with a method set.
You could type assert that value against arbitrary interfaces (which is the goal) but not the underlying type since that only exists in the runtime.
I'd also assume that reflect.TypeOf(forwardMethods(a, b)) != reflect.TypeOf(forwardMethods(a, b)) since it would have to create a new type each time.

what it's really doing is making a named type (or whatever they're called now) without a name but with a method set.

That's an interesting observation! In that sense, you could imagine a function that generalizes MakeInterface to construct an arbitrary named type with methods:
// NewTypeOf returns a new named type with the given underlying type and methods.
// The Index and Type fields of each Method are ignored and computed as they would
// be by the compiler.
// The Func field of each method must be a value of a function type,
// and the "underlying" type must be assignable to its first argument,
// which is treated as the method's receiver.
// The Name of the returned type is an arbitrary string that is not the name of
// any other type, and its PkgPath is [...?].
func NewTypeOf(underlying Type, methods []Method) Type {
	…
}

I'd also assume that reflect.TypeOf(forwardMethods(a, b)) != reflect.TypeOf(forwardMethods(a, b)) since it would have to create a new type each time.

I think they would need to be equal. I would expect them to be equal, any any rate: they are both the type "reflected instance of interface I", where I is itself the same (unnamed) interface type for both.
But we could make that explicit by adding both NewTypeOf and InterfaceOf, in which case you could store the concrete types in a sync.Map indexed by interface type.

@bcmills how does that satisfy the use-case of avoiding interface method erasure?
That let's you frankenstein a type out of two types but you need a way to say "this method forwards invocation to this value and that method forwards invocation to that other value." In addition to building the method set you need to specify the value to use as the receiver for each method separately.
That's why I was thinking it would have to be more like a closure but for types (and hence why I assumed equality would not hold).
I'm starting to think that neither reflection or generics are good solutions to this problem and there just needs to be a way in the language to embed an interface in an "open" fashion where it transparently "forwards" type asserts to unmatched interfaces, like @cep21's struct{ wrap(error) } construct.

you need a way to say "this method forwards invocation to this value and that method forwards invocation to that other value."

Presumably you'd do that the same way you would currently hand-write the wrapper: to merge N values the space-efficient way, you make a struct type with N fields, and define each method to forward the call to the appropriate field.
Or, to do it the less-code way, you define a struct type with a function field for each method, and define each method to call the function in the corresponding field.
It's admittedly quite a bit more boilerplate than a MakeInterface that synthesizes a concrete type on its own, but it's a bit more consistent with the Go language proper.

That would be some annoying code to write but otoh it would only need to be written once and everyone could use it. There would be a lot of indirection when you called a method on it. That would be fine for an error, probably less so for an http.ResponseWriter though.
Perhaps it could be implemented in the reflect package, even if the necessary primitives were all present, since it could likely cheat and do more optimizations than user code.
A pro for having it be in the language is that you could still assert to the concrete type of the wrapper should the need arise. If the type is built at runtime you'd have n versions, one for each wrapped type, and none exist at compile time.

@jimmyfrasche One option is that StructOf left open the possibility of promoting methods from fields. You could create a struct type using StructOf which embeds a concrete type, and then add methods using NewTypeOf (or whatever it's called). The problem with this is that the new "methods" being closures means that the type would essentially have static members shared by all instances. The InterfaceOf approach at least makes it so that the underlying value is the collection of values captured by the method closures.

Based on the idea that you should be able to cache pairs of types, I tried writing out a more complete implementation using NewTypeOf.
I wanted to use reflect to do the equivalent of:
type createdAtRuntime struct {
  s S
  t T
}

func (c createdAtRuntime) SomeMethodNotOnT() {
  c.s.SomeMethodNotOnT()
}

// other methods of S not on T

func (c createdAtRuntime) SomeMethodOnSandT() {
  c.t.SomeMethodOnSandT()
}

// other methods of T

That way I could cache createdAtRuntime and create new instances just by setting the fields.
The code was mostly straightforward.
Create a struct with two fields a and b of the appropriate types.
Collect the relevant methods and create proxies that call them on a or b appropriately.
I ran into a problem though.
In order to create the proxies I needed the result of NewTypeOf but of course the proxies are the methods that I want to pass to NewTypeOf. I'm not sure of a way around that. (The code for the proxies is here https://play.golang.org/p/xUSfI92a7O if it's of interest but the rest was certainly uninteresting).
Also, I'm not sure how unexported methods would work with this. If I have a "Wrap" func defined in package A and in package B I create a value from a type defined in package B and one in C, then can I assert for interfaces containing unexported methods in package B? What if I pass the value to package C?

In order to create the proxies I needed the result of NewTypeOf but of course the proxies are the methods that I want to pass to NewTypeOf.

Right, we'd need an explicit carve-out to allow the receivers of those methods to be the underlying type rather than the actual receiver type. (In my comment, I described that as 'the "underlying" type (or a pointer to that type) must be assignable to the function's first argument'.)

Also, I'm not sure how unexported methods would work with this.

I think you'd have to use embedding, which implies that NewTypeOf (and by extension StructOf) would have to handle embedded fields. (StructOf currently does not, but explicitly documents that it may in the future.)

I think we need MakeValue(methods []Method) reflect.Value with the following semantics:

the returned Value's type has an arbitrary, unique, illegal name (say "1"), an invalid PkgPath, and the specified method set
the underlying type is struct{}
when methods are called on the value they are not passed a receiver—they're treated like ordinary function calls
type assertions on unexported methods respect the PkgPath of the method not the type

That let's us create a value with an arbitrary method set and type from regular functions.
Let's also say there was a way to create a method value† from a given reflect.Value (there may be one hidden in reflect but I couldn't spot it). Just to have some notation let's say it's reflect.MethodValue(v reflect.Value, methodName string) reflect.Value where the returned Value is the function.
Then given two arbitrary Value's to combine, we just need to

collect and compute the method set,
use MethodValue to turn the methods we're keeping into functions,
create a Method for each copying the Name, PkgPath from the original Method, and setting Func to the result of MethodValue,
and pass those to MakeValue.

† I don't see this used much in the wild so if anyone reading is unfamiliar, given a type T with a method func (t T) M(a int, b string) float64 and a value of that type v then v.M is equivalent to calling the below
func methodValueOfM(t T) func(int, string) float64 {
  return func(a int, b string) float64 {
    return t.M(a, b)
  }
}

@bcmills yes, sorry—missed the point about the underlying type to break the loop. That does have the disadvantage that you couldn't create a method which invokes another method using reflection as the receiver doesn't have the method set yet. Nothing to do with the use-case under discussion, but a limitation nonetheless.
Maybe it needs to be a builder pattern where NewTypeOf(underlying Type) returns some type that has an AddMethod(reflect.Method) method and a Build() reflect.Type method? Not especially idiomatic but without separating the steps in someway I think you're always going to have some edge to hit. The piecemeal construction avoids the issue.
Back to the use-case, as far as embedding, since we're, by definition, trying to combine two types with intersecting method sets we could only embed one of the two types so you'd have the same problems just with one less type to worry about except you'd also need to worry about a method colliding the embedded types name.
But that made me realize that in order for any of this to work we'd potentially need to create the equivalent of
package x //where the CombineValues func is defined
struct {
  a y.typeName
  b z.typeName
}

since the types we're combining come in interfaces so the underlying type could very well be unexported and is likely from another package. For that to work the constructed type would have to operate by different rules than ordinary Go types, which I'm sure would cause all kinds of subtle problems.
The MakeValue approach suffers from the same issue but in a way that seems intuitively safer to me as the methods are siloed. You could do some weird stuff with MakeFunc. I think it would be easier to reason about the interactions there. But I'm just speculating.

Field names conflicting with methods is potentially unfortunate. However, the solution already exists, since Anonymous bool can be set separately from Name string (the type aliases proposal describes this as a solved problem).
As for what the receiver type would need to be for NewTypeOf methods, if it can be Value or an interface type, that would work. It is unfortunate though.
As for unexported methods, the behaviour of promotion in the language is that unexported methods are promoted, so you should be able to assert to an interface containing those methods (provided the method name in the interface and the method name on the type originate in the same package).
MakeValue could also work. An interesting potential is that your "two Values" could be
val1 := reflect.ValueOf(Wrap{ inner: err })
val2 := val1.Field(0).Elem()

@stevenblenkinsop good point about Anonymous—with that it's just a matter of generating a name longer than any methods, aaaaaaaaaa instead of just a, and so on.
The issue with the receiver type is that instead of
func (f Foo) Bar() {...}

you have to create the methods like
func (f struct{ aaaaa S; bbbbb T}) Bar()

but that should actually be fine if NewTypeOf goes through and fixes up the signatures (and it would have to) since when it's invoked the reflect.Value passed in as the receiver would be of the correct type, so my previous concern was simply due to insufficient blood-coffee levels.
The interesting thing about this wrt to unexported methods is that if you create a type from two types defined in separate packages, you could have two unexported methods on the created type with the same Name but different PkgPaths so it could, say, assert to interface{foo()} in both packages but call different methods. That is how it should behave, but it is a bit weird.
Having thought about this more I think @bcmills solution could work. I'll write up a version using it and my MakeValue later tonight to compare them and see if I run into any more issues.

Still playing with this, but I think NewTypeOf is the only way forward.
All of the other alternatives (including first class support in the language) become O(n) when composed.
If you had a function for combining two values in some way, say Combine, there's going to be cases where, for example, an error is returned, wrapped, returned, wrapped, and so on, several times resulting in something equivalent to Combine(Combine(Combine(err1, err2), err3), err4). Each layer adds another onion layer of indirection. For errors this isn't a big deal but for other uses like an http.ResponseWriter it could be a deal breaker.
However, with NewTypeOf, an implementation of Combine could detect types it created and optimize it so that there was never more than one level of indirection. That makes construction of the type more involved but that can be easily cached.

I've created a gist of a rough implementation. Warning: a little over 300 LOC.
While it's an optimizing implementation, the implementation is far from optimized. I pessimized it fairly extensively to make it easier to understand what it's doing since I can't actually run it to verify. It should be relatively straightforward to reduce allocations and the number of passes over the data, though.
To avoid creating a tree of combined values that has to proxy and re-proxy method invocations up to the depth of the tree, it adds a marker field as the 0th field of the struct it creates. When it's called on a value whose type has that marker field, like Combine(Combine(a, b), Combine(c, d)), it "flattens" everything out and computes the minimal method set as if it had been called like a hypothetical CombineVariadic(a, b, c, d). It also discards any fields, aside from the marker field, that do not contribute to the method set, allowing them to be garbage collected when they do not contribute anything.
A simpler, non-optimizing implementation would work, too, but it would have unfortunate performance implications in all but the simplest of cases making it less likely to be used.

If something like that was added to the reflect package itself, then fmt.Errorf could be rewritten to something like:
// Errorf formats according to a format specifier and returns the string
// as a value that satisfies error.
//
// The methods of the first error in the format args are still there
// [editor's note: I shouldn't write this docstring]
func Errorf(format string, a ...interface{}) error {
  msg := errors.New(Sprintf(format, a...))
  if err := firstError(a); err != nil {
    return reflect.Combine(err, msg)
  }
  return msg
}
func firstError(a []interface{}) error {
  for _, i := range a {
    if e, ok := i.(error); ok {
      return e
    }
  }
  return nil
}

That's not a light change to consider, it could have some weird consequences, but it would mean that a lot of code that hides optional methods on errors would stop doing so with no modification.

Another issue: (should have tested this earlier) StructOf can't create an unexported field, though maybe that's not an issue, since you could only get to the fields with reflect anyway and they're fairly arbitrary. If you use it to created exported fields with an unexported type from another package you can set that field, which is mildly surprising, but that is what we want to do here: https://play.golang.org/p/wQ8dC45dmI

Let's say for the moment that reflect can create new types with methods, something equivalent to what I wrote above is added to the stdlib, and that it becomes the blessed way, by convention, of composing errors.
This approach works well for errors with additional methods but not so well for errors with additional fields or sentinel errors, like io.EOF. IsX style funcs, like os.IsPermission, that dispatch on the identity of the type don't play well with this scheme.
Errors with additional fields can be made to work with this approach by adding an unexported method and an exported helper. For example, let's examine os.PathError. If it adds a method like
func (p *PathError) pathError() *PathError {
  return p
}

and the OS package adds a func like
func IsPathError(err error) *PathError { // result == nil implies false
  if p, ok := err.(interface{pathError() *PathError}); ok {
    return p.pathError()
  }
  return nil
}

then an error, under any number of combinations (with non-PathErrors), is still fully inspectable. Helpers like IsPermission could use similar tactics behind the scenes to extract the correct underlying error for their tests.
There's a small increase in API and implementation but everything works and is Go1 compatible.
Sentinel errors are a bit more annoying. They could follow the same basic approach, but each one would need its own type, like
type eof struct{}
func (e eof) Error() string { return "EOF" }
func (e eof) isEOF() {}
var EOF = eof{}
func IsEOF(err error) bool {
  _, eof := err.(interface{isEOF()})
  return eof
}

I don't think that's strictly Go1 compatible, but the only code I can think of that that would break is something using reflect. Perhaps it would be permissible as part of the transition to Go2?

It looks like the specific approach taken in the gist I posted relied on a bug ( #22073 ).
If it wants to expose unexported methods, it would need to use embedding to handle the method promotion (once that is in place) and only create proxy methods as tie-breakers for exported methods.
However that has several unhappy implications:

it cannot remove any "useless" values as it cannot detect when they do not contribute to the resultant method set
it cannot create tie-breakers for unexported methods which implicitly removes the method from the method set due to the ambiguity

1 is unfortunate but non-essential.
However, 2 means it just cannot handle unexported methods correctly. It needs to stick to only exported methods and always hide all unexported methods.
So the approach in the gist is correct and it can discard values that don't add to the method set, but it's less useful than I'd hoped because it can't handle unexported methods.
In order to handle unexported methods in the same manner as exported methods, it would need to either be a builtin or a function defined in the reflect package that circumvents the usual protections.

I hit a use case for this while working on google/go-cmp#88. The gist of the problem is that TypeOf and ValueOf can never directly return a type that is an interface. I understand why it is the case, but it goes against user expectation:
b := new(bytes.Reader)
reflect.TypeOf(b)            // *bytes.Reader
reflect.TypeOf(io.Reader(b)) // io.Reader expected, but got *bytes.Reader

This causes an issue when doing something like:
equateNotExist := cmp.Comparer(func(x, y error) bool { ... })
var e1, e2 error = syscall.ENOENT, os.ErrNotExist
cmp.Equal(e1, e2, equateNotExist) // false

The user expected this to be true since there is a custom comparer that knows how to compare error interfaces. However, this returns false because under the hood, cmp.Equal uses reflect.ValueOf and loses the fact that the user intended to pass in two error interfaces.
If you so much as pass even a pointer to the interface, then it works as expected:
cmp.Equal(&e1, &e2, equateNotExist) // true

However, doing so is not intuitive at all.
In an effort to make this more user-friendly, my approach was to try to promote the initial top-level value into an interface with the common set of methods to both values:
// Determine the set of common methods.
type method struct {
	Name string
	Type reflect.Type
}
mm := make(map[method]int)
for i := 0; i < vx.NumMethod(); i++ {
	mm[method{vx.Type().Method(i).Name, vx.Method(i).Type()}]++
}
for i := 0; i < vy.NumMethod(); i++ {
	mm[method{vy.Type().Method(i).Name, vy.Method(i).Type()}]++
}
var ms []reflect.Method
for m, n := range mm {
	if n == 2 {
		ms = append(ms, reflect.Method{Name: m.Name, Type: m.Type})
	}
}

// Promote to interface type with common method set.
iface := reflect.InterfaceOf(ms)
return vx.Convert(iface), vy.Convert(iface)

but this does not work since there is no reflect.InterfaceOf method.

@dsnet I might misread this, but the documentation says that

The comparer f must be a function "func(T, T) bool" and is implicitly filtered to input values assignable to T. If T is an interface, it is possible that f is called with two values of different concrete types that both implement T.

That would seem to cover the case you mention? In particular this works.

@Merovius, so as to keep this thread about InterfaceOf, I'll comment on the google/go-cmp#88 regarding your comment. My point in my previous comment is that I had use-case for InterfaceOf to create a dynamic interface that contains the common set of methods for 2 concrete types.

I don't want to be a party pooper, but adding a feature like reflect.MakeInterface — or any other way of creating arbitrary types to fulfill interfaces at runtime — pushes the Go complexity clock quite close to midnight.  My fear hearkens from building, maintaining, and diagnosing production Java servers that pushed the JRE though things like cglib and similar machinations.  I am really not trying to sound like reactionary, but these systems were an absolute nightmare and enabled bounds of complexity in Inversion of Control harnesses.  The only thing that could — barely — tame Java's dynamic generation is Java's superb runtime debugger ecosystem, which we don't have.
I skimmed over the issue's feedback and wondered: is there value in having a discussion whether we should have this feature at all and whether the benefits disproportionately outweigh the likely costs.

@matttproud I assume you are talking about the case of adding methods to a type created by using the reflect package.  E.g., adding methods to a type created via reflect.StructOf.  I clarify because I think that the function that does that operation should not be called reflect.MakeInterface.  I think that reflect.MakeInterface,  possibly better named reflect.InterfaceOf, is a function that creates a new interface type, just as reflect.StructOf creates a new struct type.  I don't see any particular harm in being able to do that.
Personally I'm also not too worried about being able to add methods to a type created by reflect.  That is nothing like Java's cglib.  In Go, the code for the actual methods will be, essentially, a function literal.  Adding methods to a type will amount to a different way of calling a function literal.  Yes, the stack trace will be complex, but this is still much less powerful and much less complex than cglib.

I tried to imitate reflect.StructOf by creating reflect.StructOf but I got the following error,
unexpected fault address 0xc000198090
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xc000198090 pc=0xc000198090]

Here's the file added to reflect package,
package reflect

import (
	"sort"
	"unsafe"
)

type InterfaceMethod struct {
	Name string
	Func Value
}

func InterfaceOf(funcs []InterfaceMethod) Value {
	if len(funcs) == 0 {
		return New(TypeOf("Interface {}"))
	}

	var (
		repr     = []byte("struct { /* non-invertible */ }")
		hash     = fnv1(0, []byte("struct { /* non-invertible */ }")...)
		size     uintptr
		typalign uint8
		methods  []method
	)

	// add sorted methods here
	{
		for idx := range funcs {
			if funcs[idx].Func.Type().Kind() != Func {
				panic("funcs contains non-func")
			}
			c := funcs[idx].Name[0]
			if !('A' <= c && c <= 'Z') {
				panic("funcs contains unexported func")
			}
		}

		sort.Slice(funcs, func(i, j int) bool {
			return funcs[i].Name < funcs[j].Name
		})

		for idx := range funcs {
			ft := funcs[idx].Func.Type().common()
			ft.uncommon()

			ifn := funcs[idx].Func
			methods = append(methods, method{
				name: resolveReflectName(newName(funcs[idx].Name, "", true)),
				mtyp: resolveReflectType(ft),
				ifn:  resolveReflectText(unsafe.Pointer(&ifn)),
				tfn:  resolveReflectText(unsafe.Pointer(&ifn)),
			})
		}
	}

	var typ *structType
	var ut *uncommonType

	{
		// A *rtype representing a struct is followed directly in memory by an
		// array of method objects representing the methods attached to the
		// struct. To get the same layout for a run time generated type, we
		// need an array directly following the uncommonType memory.
		// A similar strategy is used for funcTypeFixed4, ...funcTypeFixedN.
		tt := New(StructOf([]StructField{
			{Name: "S", Type: TypeOf(structType{})},
			{Name: "U", Type: TypeOf(uncommonType{})},
			{Name: "M", Type: ArrayOf(len(methods), TypeOf(methods[0]))},
		}))

		typ = (*structType)(tt.Elem().Field(0).Addr().UnsafePointer())
		ut = (*uncommonType)(tt.Elem().Field(1).Addr().UnsafePointer())

		copy(tt.Elem().Field(2).Slice(0, len(methods)).Interface().([]method), methods)
	}

	ut.mcount = uint16(len(methods))
	ut.xcount = ut.mcount // we only have exported methods
	ut.moff = uint32(unsafe.Sizeof(uncommonType{}))
	str := string(repr)

	// Round the size up to be a multiple of the alignment.
	size = align(size, uintptr(typalign))

	// Make the struct type.
	var istruct any = struct{}{}
	prototype := *(**structType)(unsafe.Pointer(&istruct))
	*typ = *prototype

	typ.str = resolveReflectName(newName(str, "", false))
	typ.tflag = 0 // TODO: set tflagRegularMemory
	typ.hash = hash
	typ.size = size
	typ.ptrdata = typeptrdata(typ.common())
	typ.align = typalign
	typ.fieldAlign = typalign
	typ.ptrToThis = 0
	typ.tflag |= tflagUncommon

	{
		typ.kind &^= kindGCProg
		bv := new(bitVector)
		addTypeBits(bv, 0, typ.common())
		if len(bv.data) > 0 {
			typ.gcdata = &bv.data[0]
		}
	}

	typ.equal = nil

	typ.kind &^= kindDirectIface

	return Zero(&typ.rtype)
}

Here's the testing code,
package main

import (
    "fmt"
    "reflect"
    "testing"
)

func TestInterfaceOf(t *testing.T) {
    type AliceInterface interface {
        BobMethod() (error, string)
    }
    Alice := reflect.TypeOf((*AliceInterface)(nil)).Elem()
    Bob := Alice.Method(0)

    Carol := reflect.MakeFunc(Bob.Type, func(args []reflect.Value) []reflect.Value {
        fmt.Println("[wow]")
        nilError := reflect.Zero(reflect.TypeOf((*error)(nil)).Elem())
        return []reflect.Value{nilError, reflect.ValueOf("haha")}
    })

    Dan := reflect.InterfaceOf([]reflect.InterfaceMethod{
        {
            Name: "BobMethod",
            Func: Carol,
        },
    })

    Dan.Method(0).Call([]reflect.Value{}) // panic

    dan := Dan.Interface().(AliceInterface)
    dan.BobMethod() // also panic

}

Does anybody know why?

Why was my comment marked as off-topic?  😠

I would guess that it's because the Go issue tracker is explicitly not for asking questions. I'd try either golang-nuts or /r/golang.


bedToBam removes last character from read names

bedToBam removes the last character from the read name. This happens in bedtools 2.29 and 2.28 but not in 2.27.1.

Can you provide a minimal example?

Likely related to https://github.com/arq5x/bedtools2/issues/762#issuecomment-528900976

echo -e "1\t1000\t2000\tread_name\t255\t+" | bedtools-2.29.0 bedtobam -i - -g /ref/chrsize.tsv | samtools view

read_nam        0       1       1001    255     1000M   *       0       0       *       *
echo -e "1\t1000\t2000\tread_name\t255\t+" | bedtools-2.27.1 bedtobam -i - -g /ref/chrsize.tsv | samtools view

read_name        0       1       1001    255     1000M   *       0       0       *       *

Thanks for reporting, likely a simple fix.

My pleasure.


[qt5-base:x64-windows] build failure

Host Environment

OS: Windows
Compiler: VS 2019 16.6.3 (latest version)

To Reproduce
Steps to reproduce the behavior:
./vcpkg upgrade --no-dry-run
Failure logs
cl : Command line warning D9025 : overriding '/Fo./' with '/Foqlibraryinfo_final.obj'
WARNING: Failure to find: C:/src/vcpkg/buildtrees/qt5-base/x64-windows-dbg/src/corelib/qt5cored_metatypes.json
WARNING: Failure to find: C:/src/vcpkg/buildtrees/qt5-base/x64-windows-dbg/src/corelib/qt5core_metatypes.json
Error: dependent 'C:\src\vcpkg\buildtrees\qt5-base\x64-windows-dbg\src\corelib\qt5cored_metatypes.json' does not exist.
jom: C:\src\vcpkg\buildtrees\qt5-base\x64-windows-dbg\src\corelib\Makefile [debug] Error 2
jom: C:\src\vcpkg\buildtrees\qt5-base\x64-windows-dbg\src\Makefile [sub-corelib-make_first] Error 2
jom: C:\src\vcpkg\buildtrees\qt5-base\x64-windows-dbg\Makefile [sub-src-make_first] Error 2

Additional context
Upgrading my vcpkg install folder today, with the QT5 port updated to 5.15 #11596 , lead to failure.
Re-launching the installation gives the same failure.
Other issues regarding latest qt5 update are also emerging from ci #12229 #11130

Other issues regarding latest qt5 update are also emerging from ci #12229 #11130

Nothing to see here. qt5-imageformats is sometimes a bit flaky and just need to be rerun and the qt5-base regression is a download failure of winflexbison.
I also get the warnings:
cl : Command line warning D9025 : overriding '/Fo./' with '/Foqlibraryinfo_final.obj'
WARNING: Failure to find: E:/qt/buildtrees/qt5-base/x64-windows-static-dbg/src/corelib/qt5cored_metatypes.json
WARNING: Failure to find: E:/qt/buildtrees/qt5-base/x64-windows-static-dbg/src/corelib/qt5core_metatypes.json
E:\qt\buildtrees\qt5-base\src\5.15.0-f256990b37.clean\src\corelib\kernel\qobjectdefs.h(0): Note: No relevant classes found. No output generated.
WARNING: Failure to find: E:/qt/buildtrees/qt5-base/x64-windows-static-dbg/src/gui/qt5guid_metatypes.json
WARNING: Failure to find: E:/qt/buildtrees/qt5-base/x64-windows-static-dbg/src/gui/qt5gui_metatypes.json
E:\qt\buildtrees\qt5-base\src\5.15.0-f256990b37.clean\src\network\access\qnetworkaccessdebugpipebackend_p.h(0): Note: No relevant classes found. No output generated.
WARNING: Failure to find: E:/qt/buildtrees/qt5-base/x64-windows-static-dbg/src/widgets/qt5widgetsd_metatypes.json
WARNING: Failure to find: E:/qt/buildtrees/qt5-base/x64-windows-static-dbg/src/widgets/qt5widgets_metatypes.json

but the build succeeds.  I also see the *.json files in the buildtree being generated. hmm...

>CMake Error at /Users/vagrant/Data/installed/x64-osx/share/cmake/Qt5Core/Qt5CoreConfig.cmake:95 (message):
  Library not found: zstd
Call Stack (most recent call first):
  /Users/vagrant/Data/installed/x64-osx/share/cmake/Qt5Core/Qt5CoreConfig.cmake:285 (_qt5_Core_process_prl_file)
  /Volumes/data/work/1/s/scripts/buildsystems/vcpkg.cmake:415 (_find_package)
  /Users/vagrant/Data/installed/x64-osx/share/cmake/Qt5Gui/Qt5GuiConfig.cmake:231 (find_package)
  /Volumes/data/work/1/s/scripts/buildsystems/vcpkg.cmake:415 (_find_package)
  /Users/vagrant/Data/installed/x64-osx/share/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake:231 (find_package)
  /Volumes/data/work/1/s/scripts/buildsystems/vcpkg.cmake:415 (_find_package)
  software/VO/AlternativeVO/CMakeLists.txt:7 (find_package)

From https://github.com/microsoft/vcpkg/pull/12229#issuecomment-656078755: Update qt5-base build-depends to include zstd. (Currently trying to find the place actually introducing zstd)

@JackBoosY why requires:repro?

@cenit Because I can't reproduce this issue on my machine.

Also happen to me:
vcpkg.exe install opencv3[qt,ipp,world,core,dnn,flann,jpeg,opengl,png,tiff,webp,tbb,ffmpeg,jasper,contrib,eigen] --triplet x64-windows
install-x64-windows-dbg-out.log
tldr (the log):
C:\Users\User\Desktop\vcpkg\buildtrees\opencv3\src\3.4.7-8b40424ca7.clean\modules\cvv\src\qtutil\../util/observer_ptr.hpp(177): error C2039: 'logic_error': is not a member of 'std'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\include\memory(24): note: see declaration of 'std'

C:\Users\User\Desktop\vcpkg\buildtrees\opencv3\src\3.4.7-8b40424ca7.clean\modules\cvv\src\qtutil\../util/observer_ptr.hpp(174): note: while compiling class template member function 'void cvv::util::ObserverPtr<QVBoxLayout>::enforceNotNull(void) const'
C:\Users\User\Desktop\vcpkg\buildtrees\opencv3\src\3.4.7-8b40424ca7.clean\modules\cvv\src\qtutil\../util/observer_ptr.hpp(96): note: see reference to function template instantiation 'void cvv::util::ObserverPtr<QVBoxLayout>::enforceNotNull(void) const' being compiled
C:\Users\User\Desktop\vcpkg\buildtrees\opencv3\src\3.4.7-8b40424ca7.clean\modules\cvv\src\qtutil\collapsable.hpp(134): note: see reference to class template instantiation 'cvv::util::ObserverPtr<QVBoxLayout>' being compiled
C:\Users\User\Desktop\vcpkg\buildtrees\opencv3\src\3.4.7-8b40424ca7.clean\modules\cvv\src\qtutil\../util/observer_ptr.hpp(177): error C2065: 'logic_error': undeclared identifier

@talregev: wrong issues? This one is about qt5-base not opencv3.

yes. I delete it.

@cenit is this still an issue? If yes could you attach both logs (err and out) ?

yes still an issue, I can't use any qt5 project on windows via vcpkg from master anymore
build-x64-windows-dbg-err.log
build-x64-windows-dbg-out.log
config-x64-windows-dbg-err.log
config-x64-windows-dbg-out.log
config-x64-windows-rel-err.log
config-x64-windows-rel-out.log
stdout-x64-windows.log

Have you tried without the symlink?
I see
IC:\Users\ssinigar\Codice\vcpkg_openmvg\buildtrees\qt5-base\src\5.15.0-f256990b37.clean\include\QtCore
And
IC:\src\vcpkg_openmvg\buildtrees\qt5-base\src\5.15.0-f256990b37.clean/include\QtCore
The first path is also very long.

I launched the build from C:\src, which is a subst of C:\Users\ssinigar\Codice, I don't know how it also expanded it to the full path. Also, no problem with any other port which has the infamous maximum path length
This also was a supported way, and it's very difficult also to propagate any change to it

How did you subst that? I always get a "invalid paramter error C:\src" if i try to subst C:\src <pathtosubst>

after testing your mklink is the reason it fails. If you directly use the base path it works. If you truly use a subst it also works.  If you use mklink as a junction /J instead of a normal symlink it also works. So the symbolic link is the issue.

Let me investigate my own setup then. Thanks for taking time to already test many configs

@cenit: I assume you found a solution? (switching to nmake can also solve the problem but I only tested that manually outside the portfile which than run into other problems since the env was not correctly setup.)

Yes it was a symlink and not a junction. Strangely enough it fails mixing the two paths.  No output generated.


[Airbrake] [Production] Unexpected token '<'

Airbrake error: #2021
Airbrake project: datanator_frontend
Error type: SyntaxError
Error message: Unexpected token '<'
Where: <no information>
Occurred at: Jun 17, 2020 07:01:39 UTC
First seen at: Jun 17, 2020 07:01:39 UTC
Occurrences: 0 (0 since last deploy on Jun 17, 2020 02:15:06 UTC)
Severity: error
URL: https://datanator.info/
File: SyntaxError: Unexpected token '<'
Backtrace:
SyntaxError: Unexpected token '<':0:in

I think this is an error in JSON content


ffmpeg: add support of specifying Android ABI version when building FFmpeg

Thanks for updating the script! I will start the merge process.


event-protocol: Remove series and timestamp

Summary
Remove series and timestamp properties from all events.
Details
Modified the schema
Motivation and Context
This aligns Gherkin generated events (source and pickle) with the event-protocol validator.
Gherkin-created pickle events currently do not have those properties. I don't think we have any consumers that require them at the moment.
We can add them back when/if a consumer needs them. I'd rather not add them prematurely, Updating Gherkin in 8 languages is a lot of work.
How Has This Been Tested?
../../gherkin/javascript/bin/gherkin ../../gherkin/testdata/good/*.feature --no-ast | ./validate.js

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue).
[ ] New feature (non-breaking change which adds functionality).
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist:

[ ] I've added tests for my code.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.

The GUi spike uses the timestamp from the start message but not for the rest. The series isn't used yet, but I have immediate plans to use it. I think it's a good idea. Let's just make it optional.

If a property is optional, what would you do if the property is absent? I think optional properties should be avoided in general.
I think series is superfluous because there are other mechanisms available to group related events. The most obvious one being a session, or a stream: Every time someone connects and pumps events, they can be considered related.
For timestamp - could you just generate a timestamp when you receive the event?

Agreed about series - I'll try that out in the GUI spike.
I think that for the start event, it makes sense to allow the sender to put out extra metadata in the event if they want to. Especially while we figure out the finer details of the protocol. The listeners just need to be robust enough to handle those optional parameters being absent.
timestamp would be useful I think, for example if we allow results to be cached locally while the client is off the internet. But if it's awkward to send from some senders, we can make it optional, and use the receipt timestamp as a fallback.
Another example is workingDirectory. It will certainly be useful to have this for the GUI, but I'm not sure we need to make it mandatory.

I'm gonna merge this, and we can talk about adding the timestamp and other optional params back in separately.

\o/


recipes-core: containers-tags: update tags for scarthgap

Summary of changes:

We are not appending anything to container tags anymore;
We don't need vivante specific tags anymore because every platform shares the same major;
We don't use portainer anymore;
TCB is not supposed to be run from within Torizon so it's unneeded;
Remove duplicated variables;
dotnet-uno was replaced by dotnet-wayland;
We have only one dotnet8-debug image now.

Related-to: TCCP-840, TCCP-841, TCCP-706

Nice cleanup in the file!
@charles2910, could you also, please, cherry-pick this PR to our master branch?

Nice cleanup in the file! @charles2910, could you also, please, cherry-pick this PR to our master branch?

There we go: https://github.com/torizon/meta-toradex-torizon/pull/158

Hey y'all, I've drafted this because this is not released yet. Not that it makes a big difference, but I don't want to redo commits because we missed something or our side, so please wait until we get our stuff together for the release. Thanks!


fix: http return int64 for int column

fixes: #28
Have added a check for the type of value in the Next method, it will only cast true integers to int64 which have been set as float64 as a bug.
Will it be sufficient to handle all the cases? I am new to the repository, so I have tested with a local connection to the libsql database and it does convert the value to int.
Let me know for any other improvements that can be added, or is this a valid fix?
Thank you

Hi @Mr-Destructive, thank you for your contribution but this is not the right way of fixing the problem. To fix it properly we have to look on the expected type of result and cast accordingly.


Allow "this" value to be set for subscriptions

Support for setting the "this" values that functions should be invoked with when subscriptions are dispatched.
Support for removing a function bound to a given "this" value.
Removed "addOnce" method and turned its behavior (along with "this"/context and priority) into an option that is passed to "add".
README updates for 1-3.

The omission of this feature is intentional (just pass a closure), and I'm not really looking to have this API arbitrarily refactored.

I was thinking in terms of attaching/detaching prose mirror plugins. It would be cleaner and less laborious if you could just pass a function that is shared amongst all plugin instances with the context set to the current plugin instance to subscription.add/remove instead of having to create a new closure or bound version of the function each time. But ok, point taken.


The "Rubric" is not showing in articles imported from golfdigest.com.

This should probably be imported as metadata.

Rubric is displayed on this page now https://sportsmagazine-poc--helix-sportsmagazine--headwirecom.hlx.page/import-test/the-loop/_default/article/2022/11/jordan-poole-golden-state-warriors-puppy
Other pages need to be re-imported.
Fonts need to be fixed in general.

See previous comment


local statement not handled

In the example below, local variable y is marked as global:
julia> solve(:(function f(x)
           local y
           @sync @async y = 1
           x + y
       end))
[]function (@global f)(@x)
    #= REPL[42]:2 =#
    local @global y
    #= REPL[42]:3 =#
    #= REPL[42]:3 =# @sync #= REPL[42]:3 =# @async(y = 1)
    #= REPL[42]:4 =#
    (@global +)(@x, @global y)
end

Hi, thanks for this feedback!
The macrocalls prevent inference. I think it not feasible to support all macrocalls...
julia> solve(:(function f(x)
                  local y
                  y = 1
                  x + y
              end))
[]function (@global f)(@x)
    #= REPL[14]:2 =#
    local @y
    #= REPL[14]:3 =#
    @y = 1
    #= REPL[14]:4 =#
    (@global +)(@x, @y)
end

I don't think the macro is the problem. See:
julia> solve(:(function f(x)
           local y
           x + y
       end))
[]function (@global f)(@x)
    #= REPL[38]:2 =#
    local @global y
    #= REPL[38]:3 =#
    (@global +)(@x, @global y)
end

This function throws at runtime, even though the global variable y exist:
julia> function f1(x)
           local y
           x + y
       end
f1 (generic function with 1 method)

julia> function f2(x)
           x + y
       end
f2 (generic function with 1 method)

julia> y = 1
1

julia> f1(1)
ERROR: UndefVarError: y not defined
Stacktrace:
 [1] f1(::Int64) at ./REPL[43]:3
 [2] top-level scope at REPL[46]:1

julia> f2(1)
2

So, I think it would be better to pick up the scoping information just from local (treat it as an assignment, I guess).

Yes, this should be bug.

this is already fixed.


Revert "[SPARK-34739][SQL] Support add/subtract of a year-month interval to/from a timestamp"

What changes were proposed in this pull request?
Revert the commit https://github.com/apache/spark/commit/9809a2f1c5187205c81542dbdc84b71db535f6e1
Why are the changes needed?
It causes a build failure for scala 2.13.
Does this PR introduce any user-facing change?
No
How was this patch tested?
By running GA

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40725/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136143/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40725/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40725/

This GA still fails:
Build and test / Scala 2.13 build with SBT (pull_request) Failing after 7m — Scala 2.13 build with SBT

So, the issue is not related to the commit. I am closing the PR.


fix: expose server zone enum

Summary
Bring back ServerZone enum for ampli compatibility
Checklist

[x] Does your PR title have the correct title format?
Does your PR have a breaking change?:  No

@justin-fiedler @kevinpagtakhan

for ampli compatibility

What compatibility case should be supported?
Is it the only incompatibility?
Will new codegen v3 be added to support new 2.x SDK version? If yes, will codegen use new ServerZoneType type (actually constants US/EU)?

Hi @falconandy! I removed customer facing enums which includes ServerZone for Browser SDK 2.0. I then noticed that ServerZone is used by ampli v2. @justin-fiedler and I agreed that we should make Browser SDK 2.0 compatible with ampli v2, so it does not force us to work on an ampli v2. The work here is to bring back ServerZone enum but only for ampli v2 purposes.


web3.exceptions.ContractLogicError: execution reverted: please approve spending token

Hi,   could you help share some ideas about this below issues. thank you .
==code==
# upload nft metadata
meta_url = api.upload_nft_metadata(wallet_address, filename, nft_uri, tx_hash, file_size)['data']['ipfs_url']
==code==
File "D:\BlueDoc\DeepDiary\Backend\env\lib\site-packages\web3\manager.py", line 170, in formatted_response
apply_error_formatters(error_formatters, response)
File "D:\BlueDoc\DeepDiary\Backend\env\lib\site-packages\web3\manager.py", line 70, in apply_error_formatters
formatted_resp = pipe(response, error_formatters)
File "cytoolz\functoolz.pyx", line 666, in cytoolz.functoolz.pipe
File "cytoolz\functoolz.pyx", line 641, in cytoolz.functoolz.c_pipe
File "D:\BlueDoc\DeepDiary\Backend\env\lib\site-packages\web3_utils\method_formatters.py", line 576, in raise_solidity_error_on_revert
raise ContractLogicError(response['error']['message'])
web3.exceptions.ContractLogicError: execution reverted: please approve spending token

Usdc approve needs to be performed one step before payment.
for example:
def test_approve_usdc():
w3_api = ContractAPI(web3_api)
w3_api.approve_usdc(wallet_address,
private_key, "100")


SLU01, Learning Notebook typo

SLU01, Learning Notebook typo:

Ty for opening an issue! This is to be fixed in the next Prep Course edition!


docs(readme): add development instructions and update authors

Changelog
All notable changes to this project will be documented in this file.
[unreleased]
Documentation

(changelog) Update readme
(readme) Add development instructions and update authors

By the way, the Development section of the readme is getting quite long, would it be a good idea to move it to a separate file, for example development.md inside a docs directory (gibsonify/docs/development.md) and then just add a link to that file from the Development section in the Readme?
@akabla @sshazril

By the way, the Development section of the readme is getting quite long, would it be a good idea to move it to a separate file, for example development.md inside a docs directory (gibsonify/docs/development.md) and then just add a link to that file from the Development section in the Readme?
@akabla @sshazril

I'd say its up to personal preference, I've seen some pretty lengthy readmes on projects and some short ones too. That said, I'd support the idea of moving it to a different file :)


Forbid unsafe code

No unsafe code seems to be needed, so we can get a green line in cargo-geiger with #![forbid(unsafe_code)].

Thanks!


Fix FF bug where overflowed flex items are cut off

Instead of only applying 'min-height: 0' on the flex container when flex-direction is column, it always gets applied
For more info on this bug, see:
https://github.com/philipwalton/flexbugs\#flexbug-1
https://bugs.chromium.org/p/chromium/issues/detail\?id\=580196
Screenshots
Before (does not scroll all the way to the bottom of the content)

After (does scroll all the way to the bottom of the content)

@mperrotti , where can I test this in the ui-kit?

@GeorgiSTodorov - I was just testing in Kommander, but we can pair to write a story that would test this in ui-kit if you prefer

@mperrotti , I think testing it in Kommander is good enough, especially since the bug was discovered there.


Update ex1.java

Now saves data before calculating geometric means

Please only make one merge request for all your files in the future.


Support modern Flexmark

Scalatest recently switched from pegdown to flexmark.
Besides a lack of documentation, as discussed in #1736, scalatest currently works with a small range of flexmark versions.
I propose to:

add support for the more recent versions of Flexmark;
possibly, declare Flexmark as a runtime dependency, so that users do not need to guess what's needed in order to make scalatest reporting system work;
fail more clearly: the current error message is An exception or error caused a run to abort. This may have been caused by a problematic custom reporter.: it's truthful, but not particularly useful for resolution.

It is likely that the former two points make the third way less important, however my feeling is that the third is a low hanging fruit, while there might be need for more work to address the former.

Ideally, this should be a plugin dependency so it doesn’t interfere with project dependencies. It should definitely be documented.

Just ran into this issue trying to integrate with https://github.com/pact-foundation/pact-jvm on jdk8 (maybe modules makes it less of an issue on jdk9+, not sure) and which has a dependency on flexmark 0.62.2.
@DanySK where did you get to investigating an upgrade?

@caoilte I did not even start trying to upgrade scalatest, frankly. I just investigated where the issue was from. Worst case, I'll switch to kotest, Scala is becoming more and more marginal on most of my projects.

Hi, fyi I submitted a PR to update flexmark version to 0.62.2:
https://github.com/scalatest/scalatest/pull/2050

@cheeseng Hey nice! I was experimenting with pretty much the same changes using a locally built snapshot over the weekend. Weirdly, I found that even though the tests passed I couldn't use the snapshot (got runtime class not found) unless I added an extra dependency. Specifically,
"com.vladsch.flexmark" % "flexmark-profile-pegdown" % "0.62.2"
Would be interesting to know if you were also able to test locally?

@caoilte I didn't try local snapshot, the tests should be using HtmlReporter to generate the html report under target/html.  Afaik, to use local snapshot (even the real release build) we need to explicitly add the flexmark dependency like the following:
"com.vladsch.flexmark" % "flexmark-all" % "0.62.2"

The reason I think is the flexmark dependency is marked as "optional", here:
https://github.com/scalatest/scalatest/blob/3.2.x-new/project/scalatest.scala#L178
Your import does seems to be more specific than flexmark-all though, I wonder if scalatest should use flexmark-profile-pegdown instead.

@cheeseng Just checked and you're right. It also works with flexmark-all and that makes sense because every project I know running scalatest 3.1+ has to manually add that dependency (ie https://github.com/scalatest/scalatest/issues/1736).
It would certainly be interesting to explore whether profile-pegdown is a better choice overall.


feat: added cleanup of generated quay repos and robot accounts

Description
Added a CleanupQuay function to magefile which deletes quay repos and robot accounts with prefixes e2e-demos and has-e2e.
Repos get deleted only if they have never been modified and robot accounts have to be older than 24 hours to be deleted.
Quay functions have been taken from https://github.com/redhat-appstudio/image-controller/tree/main/pkg/quay
Issue ticket number and link
https://issues.redhat.com/browse/STONEBLD-1089
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
This has been tested by creating mock repos and robot accounts in my own organization and running the function on that.
Also tested via mocking in utils_test.go.
Passed in OpenShift
https://github.com/openshift/release/pull/38388
Checklist:

[X] I have performed a self-review of my code
[X] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added meaningful description with JIRA/GitHub issue key(if applicable), for example HASSuiteDescribe("STONE-123456789 devfile source")
[ ] I have updated labels (if needed)

/retest-required

/retest

/retest


Add parameters for BatchNorm

This is PR for issue #65. With this commit, FL BatchNorm can receive runningMean and runningVar additionally. This is useful when loading weights from other frameworks, etc.

@jacobkahn @vineelpratap Any updates for this?

@jacobkahn @vineelpratap @jcai1 ping!

@jacobkahn I updated the commits including your review, and rebased it on current master. However I didn't test this - I don't have test environment now. Could you review this?

@minty99 — the changes look fine, but we'd want to add some tests before merging.

@jacobkahn Do you have some guideline for adding tests? I think I should edit tests/nn/ModuleTest.cpp.

@minty99 — that's exactly right; those tests should go in ModuleTest.cpp.