gorouter doesn't support all defined cipher suites?

I have deployed CF 252 without ha_proxy on an Openstack dev landscape. I was checking ssl certificates support returned by gorouter and I don't understand why I'm getting only a couple of them instead of all.
  router:
    secure_cookies: true
    ssl_skip_validation: true
    enable_ssl: true
    cipher_suites: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA:TLS_RSA_WITH_AES_128_CBC_SHA:TLS_RSA_WITH_AES_256_CBC_SHA"
    ssl_cert: "..."
    ssl_key: "..."

output of sslscan
    Failed    TLSv1  256 bits  ECDHE-RSA-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  ECDHE-ECDSA-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  ECDHE-RSA-AES256-SHA384
    Failed    TLSv1  256 bits  ECDHE-ECDSA-AES256-SHA384
    Accepted  TLSv1  256 bits  ECDHE-RSA-AES256-SHA
    Rejected  TLSv1  256 bits  ECDHE-ECDSA-AES256-SHA
    Failed    TLSv1  256 bits  SRP-DSS-AES-256-CBC-SHA
    Failed    TLSv1  256 bits  SRP-RSA-AES-256-CBC-SHA
    Failed    TLSv1  256 bits  SRP-AES-256-CBC-SHA
    Failed    TLSv1  256 bits  DHE-DSS-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  DHE-RSA-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  DHE-RSA-AES256-SHA256
    Failed    TLSv1  256 bits  DHE-DSS-AES256-SHA256
    Rejected  TLSv1  256 bits  DHE-RSA-AES256-SHA
    Rejected  TLSv1  256 bits  DHE-DSS-AES256-SHA
    Rejected  TLSv1  256 bits  DHE-RSA-CAMELLIA256-SHA
    Rejected  TLSv1  256 bits  DHE-DSS-CAMELLIA256-SHA
    Rejected  TLSv1  256 bits  AECDH-AES256-SHA
    Failed    TLSv1  256 bits  ADH-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  ADH-AES256-SHA256
    Rejected  TLSv1  256 bits  ADH-AES256-SHA
    Rejected  TLSv1  256 bits  ADH-CAMELLIA256-SHA
    Failed    TLSv1  256 bits  ECDH-RSA-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  ECDH-ECDSA-AES256-GCM-SHA384
    Failed    TLSv1  256 bits  ECDH-RSA-AES256-SHA384
    Failed    TLSv1  256 bits  ECDH-ECDSA-AES256-SHA384
    Rejected  TLSv1  256 bits  ECDH-RSA-AES256-SHA
    Rejected  TLSv1  256 bits  ECDH-ECDSA-AES256-SHA
    Failed    TLSv1  256 bits  AES256-GCM-SHA384
    Failed    TLSv1  256 bits  AES256-SHA256
    Accepted  TLSv1  256 bits  AES256-SHA
    Rejected  TLSv1  256 bits  CAMELLIA256-SHA
    Failed    TLSv1  256 bits  PSK-AES256-CBC-SHA
    Rejected  TLSv1  168 bits  ECDHE-RSA-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  ECDHE-ECDSA-DES-CBC3-SHA
    Failed    TLSv1  168 bits  SRP-DSS-3DES-EDE-CBC-SHA
    Failed    TLSv1  168 bits  SRP-RSA-3DES-EDE-CBC-SHA
    Failed    TLSv1  168 bits  SRP-3DES-EDE-CBC-SHA
    Rejected  TLSv1  168 bits  EDH-RSA-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  EDH-DSS-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  AECDH-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  ADH-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  ECDH-RSA-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  ECDH-ECDSA-DES-CBC3-SHA
    Rejected  TLSv1  168 bits  DES-CBC3-SHA
    Failed    TLSv1  168 bits  PSK-3DES-EDE-CBC-SHA
    Failed    TLSv1  128 bits  ECDHE-RSA-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  ECDHE-ECDSA-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  ECDHE-RSA-AES128-SHA256
    Failed    TLSv1  128 bits  ECDHE-ECDSA-AES128-SHA256
    Accepted  TLSv1  128 bits  ECDHE-RSA-AES128-SHA
    Rejected  TLSv1  128 bits  ECDHE-ECDSA-AES128-SHA
    Failed    TLSv1  128 bits  SRP-DSS-AES-128-CBC-SHA
    Failed    TLSv1  128 bits  SRP-RSA-AES-128-CBC-SHA
    Failed    TLSv1  128 bits  SRP-AES-128-CBC-SHA
    Failed    TLSv1  128 bits  DHE-DSS-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  DHE-RSA-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  DHE-RSA-AES128-SHA256
    Failed    TLSv1  128 bits  DHE-DSS-AES128-SHA256
    Rejected  TLSv1  128 bits  DHE-RSA-AES128-SHA
    Rejected  TLSv1  128 bits  DHE-DSS-AES128-SHA
    Rejected  TLSv1  128 bits  DHE-RSA-SEED-SHA
    Rejected  TLSv1  128 bits  DHE-DSS-SEED-SHA
    Rejected  TLSv1  128 bits  DHE-RSA-CAMELLIA128-SHA
    Rejected  TLSv1  128 bits  DHE-DSS-CAMELLIA128-SHA
    Rejected  TLSv1  128 bits  AECDH-AES128-SHA
    Failed    TLSv1  128 bits  ADH-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  ADH-AES128-SHA256
    Rejected  TLSv1  128 bits  ADH-AES128-SHA
    Rejected  TLSv1  128 bits  ADH-SEED-SHA
    Rejected  TLSv1  128 bits  ADH-CAMELLIA128-SHA
    Failed    TLSv1  128 bits  ECDH-RSA-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  ECDH-ECDSA-AES128-GCM-SHA256
    Failed    TLSv1  128 bits  ECDH-RSA-AES128-SHA256
    Failed    TLSv1  128 bits  ECDH-ECDSA-AES128-SHA256
    Rejected  TLSv1  128 bits  ECDH-RSA-AES128-SHA
    Rejected  TLSv1  128 bits  ECDH-ECDSA-AES128-SHA
    Failed    TLSv1  128 bits  AES128-GCM-SHA256
    Failed    TLSv1  128 bits  AES128-SHA256
    Accepted  TLSv1  128 bits  AES128-SHA
    Rejected  TLSv1  128 bits  SEED-SHA
    Rejected  TLSv1  128 bits  CAMELLIA128-SHA
    Failed    TLSv1  128 bits  PSK-AES128-CBC-SHA
    Rejected  TLSv1  128 bits  ECDHE-RSA-RC4-SHA
    Rejected  TLSv1  128 bits  ECDHE-ECDSA-RC4-SHA
    Rejected  TLSv1  128 bits  AECDH-RC4-SHA
    Rejected  TLSv1  128 bits  ADH-RC4-MD5
    Rejected  TLSv1  128 bits  ECDH-RSA-RC4-SHA
    Rejected  TLSv1  128 bits  ECDH-ECDSA-RC4-SHA
    Rejected  TLSv1  128 bits  RC4-SHA
    Rejected  TLSv1  128 bits  RC4-MD5
    Failed    TLSv1  128 bits  PSK-RC4-SHA
    Rejected  TLSv1  56 bits   EDH-RSA-DES-CBC-SHA
    Rejected  TLSv1  56 bits   EDH-DSS-DES-CBC-SHA
    Rejected  TLSv1  56 bits   ADH-DES-CBC-SHA
    Rejected  TLSv1  56 bits   DES-CBC-SHA
    Rejected  TLSv1  40 bits   EXP-EDH-RSA-DES-CBC-SHA
    Rejected  TLSv1  40 bits   EXP-EDH-DSS-DES-CBC-SHA
    Rejected  TLSv1  40 bits   EXP-ADH-DES-CBC-SHA
    Rejected  TLSv1  40 bits   EXP-DES-CBC-SHA
    Rejected  TLSv1  40 bits   EXP-RC2-CBC-MD5
    Rejected  TLSv1  40 bits   EXP-ADH-RC4-MD5
    Rejected  TLSv1  40 bits   EXP-RC4-MD5
    Rejected  TLSv1  0 bits    ECDHE-RSA-NULL-SHA
    Rejected  TLSv1  0 bits    ECDHE-ECDSA-NULL-SHA
    Rejected  TLSv1  0 bits    AECDH-NULL-SHA
    Rejected  TLSv1  0 bits    ECDH-RSA-NULL-SHA
    Rejected  TLSv1  0 bits    ECDH-ECDSA-NULL-SHA
    Failed    TLSv1  0 bits    NULL-SHA256
    Rejected  TLSv1  0 bits    NULL-SHA
    Rejected  TLSv1  0 bits    NULL-MD5

output of nmap --script ssl-cert,ssl-enum-ciphers -p 443 $host
| Public Key type: rsa
| Public Key bits: 4096
| Not valid before: 2016-11-29T15:34:54+00:00
| Not valid after:  2044-04-16T15:34:54+00:00
| MD5:   a7bb e184 4b64 9b9b 8c87 147d 3561 2260
|_SHA-1: af5d 385d b225 022a 3f49 4efb 9cdb 5b56 942e de06
| ssl-enum-ciphers:
|   TLSv1.0:
|     ciphers:
|       TLS_RSA_WITH_AES_128_CBC_SHA - strong
|       TLS_RSA_WITH_AES_256_CBC_SHA - strong
|     compressors:
|       NULL
|   TLSv1.1:
|     ciphers:
|       TLS_RSA_WITH_AES_128_CBC_SHA - strong
|       TLS_RSA_WITH_AES_256_CBC_SHA - strong
|     compressors:
|       NULL
|   TLSv1.2:
|     ciphers:
|       TLS_RSA_WITH_AES_128_CBC_SHA - strong
|       TLS_RSA_WITH_AES_256_CBC_SHA - strong
|     compressors:
|       NULL
|_  least strength: strong

Hello @paolostivanin
Is your concern that the supported cipher suites don't match the default list for  router.cipher_suites, or are you asking why we don't support more by default?
We have not supported all cipher suites because most are considered insecure by our security team. In cf-release v253, the default cipher suites will be reduced to two for the same reason:
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
If your clients require additional cipher suites, you can certainly use the router.cipher_suites field to add them.

Hello @shalako,
thanks for your reply but  what I was asking is why I get only 2 supported certs when I have configured all of them in the manifest. Is that a bug or something wrong on my side?

The ciphers supported by golang crypto/tls, on which the router depends, are listed here:
https://golang.org/pkg/crypto/tls/#pkg-constants
Is it possible you've specified some unsupported ciphers?

On Feb 28, 2017, at 12:30 PM, Paolo Stivanin<EMAIL_ADDRESS>wrote:
Hello @shalako,
thanks for your reply but what I was asking is why I get only 2 supported certs when I have configured all of them in the manifest. Is that a bug or something wrong on my side?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I have specified the ciphers that are listed here:
https://github.com/cloudfoundry-incubator/routing-release/blob/661c5bc41d32f62d52ffe84dc206172e6938403b/jobs/gorouter/spec#L88
On February 28, 2017 22:36:51 Shannon Coen<EMAIL_ADDRESS>wrote:

The ciphers supported by golang crypto/tls, on which the router depends,
are listed here:
https://golang.org/pkg/crypto/tls/#pkg-constants
Is it possible you've specified some unsupported ciphers?

On Feb 28, 2017, at 12:30 PM, Paolo Stivanin<EMAIL_ADDRESS>wrote:
Hello @shalako,
thanks for your reply but what I was asking is why I get only 2 supported
certs when I have configured all of them in the manifest. Is that a bug or
something wrong on my side?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/cloudfoundry/gorouter/issues/165#issuecomment-283167538

I believe you are saying that the former default cipher suites are not actually supported in that version.
We will investigate.

@shalako yes, exactly.
Thanks!

@paolostivanin
We have verified that all the default cipher suites should work on the Gorouter.
As of CF 252 we only supported the default cipher suites.
There are two reasons why some of the defaults may not be working for you.

Some of the cipher suites require a ECDSA key pair to be configured, so even if the cipher suite is in the default, you will not be able to communicate as the Gorouter most likely is using a RSA key pair for its ssl configuration.
Another reason could be if you have a loadbalancer that is pointing toward the gorouter. That load balancer may have its own set of cipher suites that it supports.

In the future we plan to better document what cipher suites we do support.
Please let us know if there are any other questions you still have regarding the cipher suites.
Thanks,
@swetharepakula && @jberkhahn

Ok, thanks for your answer :)


Graphics Support via VJET

Opening a thread on VJET graphics support for @Wuerfel21.

You mean the VJET polygon drawing stuff? Yes, it is quite similar in concept to the FTDI EVE hardware, just significantly less powerful - with the 4 rasterizer cogs @80MHz I usually use, that is. 117 MHz is a good bit faster and you may be able to use 5 or 6 cogs. Also, the display list doesn't contain triangles/lines/polygons per se, it contains "trapezoid stacks", essentially a representation of a y-monotonous polygon as set of trapezoids defined by height and left/right slope. If you've ever written a triangle/poly fill algorithm, you probably understand what I mean. There's some computation involved in generating these from a polygon (triangles as a special case are quite simple though).

Thanks I've kept 3 base addresses free in the UX Module design. 0x80 is for the ACIA emulation so it will work with the standard RC2014 software. Another is to separate the serial Tx/Rx onto another "TTY" port.
The final base address is to build a graphics device. It might be TMS9918 like, or better I'd prefer it to be based on VJET like solution. Ideally it would work to the same VGA memory as the HiRes VGA driver. A bit like a graphic overlay.

Running both hires text and any sort of graphics at the same time is just not going to happen. However, if you're OK with switching between highres text and lowres text + graphics, it'll work out. One cog is needed to drive the VGA from line buffers, leaving 3 render cogs. 3 might be ok at 108 MHz, but 4 is really the sweet spot. If we can get rid of that pesky keyboard cog that spends of it's time doing absolutely nothing, that'd be 4 render cogs. Just need to find some other place to stuff the PS/2 bitbanging and then the scancode conversion and stuff can be just be done in Spin.

Anyways, the terminal emulation can be incorporated into the graphics rendering by simply placing some text commands into the display list that point into the terminal buffer. Can be an overlay or an underlay depending on where the commands are inserted. However, the code handling the terminal buffer will need to be adjusted for smaller screen size. VJET as-is runs in 256x238 with non-square pixels. Could be bumped to 256x240 (The reason it's 238 is because I couldn't get my code to do 240 lines and it literally makes zero difference on TV due to overscan, so I didn't bother fixing it) If we assume most of that isn't overscanned, you can reasonably get 40x28 characters (each character being 6x8, TMS9918 style). (However, I think to actually render a screen full of text, it will need 4 render cogs.) Also, it'd really help if the terminal emulation would automagically zero-terminate each line of display memory instead of just having spaces till the end.

I'm actually more inclined to close the serial interface cog (at least when development is done) as a preference, as the keyboard needs to be "fast" (in its own cog?) to allow gaming control.
And, absolutely no problem to make the video modal (lowres text + graphics) and highres.
I'm going to focus on getting the highres text working, as that's just a matter of matching terminal emulation codes and keyboard scanning. This part is critical for CP/M and general usage. The graphics part is a "stretch"
I'd like to have a command processor, so that the nice GL language can exist. It might be useful without the Z80 bus interface for other people too, so I wouldn't want the Z80 to know too much.
The EVE hardware has 32bit GL commands which are executed out of an on chip buffer. The command buffer is rendered line by line for display. I'm not sure but, I'm imagining a cog rendering a slice of the screen, and then displaying it. With 4 cogs sharing the render / display it would be real dance. It looks like the VJET geometry processing code is quite similar in function.

Yeah, it is like that a bit, the main difference is that the high-level commands have to be pre-chewed by the geometry code. The display list also has space for temp values in it used to stregnth-reduce some multiplications out of the rendering cogs.

Adding some macro commands to draw common GUI elements is definitely very easy to do.

Other thing is: What is filling the display list? Letting the Z80 write it directly would not only be quite slow, but would also require the Z80 code to know about hub RAM layout (and if it messed up, the render cogs will spill their temp values all over and cause a really bad time). So instead it probably should be a command interface. If the bus interface code can just dump whatever data arrives at its port to the Spin code, that could handle generating the display list from commands, similar to the spin interface existing VJET geometry processing code. Problem with that is that normal spin is kinda slow at doing that. Flexspin compiles to LMM assembler and is a lot faster, but it'll make your code size explode and as such requires careful optimization to fit everything. Also when you use it, you'll find and report a bug every 2 days, it's inevitable :)
I also do think that the geometry code (especially the polygon function) could be optimized more. Or if flexspin is used, anyways, custom inline ASM could also help.

Yeah closing the serial interface also works (you could even make it an option to leave it running at the expense of reduced rendering cogs) and is very much the easier option. However, do know that moving the keyboard handling somewhere else would not make it measurably slower - the reason it generally is handled in it's own cog is because the keyboard controls the data transfer speed and thus the the pins have to be monitored constantly. Compare to something like a SNES controller, where the speed is controlled entirely by the host - that is generally read once per frame in a simple spin routine.

I have put up a video of a "first draft" for the HiRes Text mode.
Excuse the slow Basic programming.
User Experience Module for RC2014
The UX Module has a major port (0x40, 0x80, 0xC0), and a minor port at one higher. I'm thinking the Major port can be a Command/Status port (like the ACIA), and the Minor port can be for 4x 8bit data.
It would be easy to write Z80 assembly to upload 32-bit longs in LSB->MSB order from the Z80 to Parallax. If the Display Commands were assumed always 32bit, then parsing the command data would become simpler too.
And, If I can dig into the EVE DL Commands a bit more then it might even be possible to replicate them (to some extent).
More when the next PCB is available.

I'm still unhappy with the Z80 bus interface solution. Works with a small MS Basic (classic) configuration, but is unreliable with larger configurations.
I've ordered some 200pf capacitors, and digital (pre biased) transistors to experiment some more.
I'm suspicious of the data bus being too slow, because unreliable behaviour seems to be bit flipping (or not settling) on the bus.
I'm also working on VT100 codes. Anna has collected VT100 games that I'll be "forced" to play for testing.

I went down the path of turning the ACIA interface into an object that could be replicated in multiple cogs, and then spent some time trying to understand why it didn't work. Long story short, there is only one /WAIT line that is ORed, so /WAIT could never go low.
There's no point trying to build multiple simultaneous interfaces to the Z80. Either the ACIA emulation is running, or the VJET interface is running. A modal state switch is required with keyboard, serial, and Z80 interface inputs.
I couldn't find a VJET with VGA interface. This OBEX VGA64 driver looks imho the closest, so I've put it in as a template.
Probably the first step to achieve is to draw a polygon on VGA, without worrying about Z80 interfacing.

Oh yes indeed, there's no VGA output driver yet. The composite one was basically borrowed from regular JET and then filed down, but the VGA one is such spaghetti (due to having to support horizontal scrolling) that I wouldn't want to use it for VJET.
Can the graphics ports not be handled by  the ACIA cog, too?

Yes, certainly the Z80 interface can be handled by one cog with no issue. I was previously hoping that I could just clone instances of ACIA code, to make debugging easy. Leider nicht.
So my job will be to write a new Z80 interface that looks like an ACIA on one set of ports, and has additional features on the other ports. It can only run in one cog because of the /WAIT control factor, but that won't be a problem, I'm sure.
The way the UX Module hardware is set up, there are only two ports 0xC0 and 0xC1 that can be used by the VJET interface.
If I take the example of the SIO/2 interface, then the port 0 is used partly as a register write pointer (3 bits) and write control bits, and read status port. The port 1 is then the data input and output port, and loads the register set by the write pointer control. There are 7 registers (from memory). For the VJET interface, we could also read additional status bits from port 1 because there's no input data per se.
So ideally, the interface would auto-increment to load polygons or pixels, or whatever needed, once set up with a x,y pixel origin. But I'm not sure what is needed, so just guessing now.

Also just noticed that the VJET files need to be ASCII'd, oof.
Eitherhow, will probably get the VGA backend going soon™. Right now I've got a shiny new P2 on my desk, so I'll toy with that for today. First order of business will be getting VJET going on that (Spin Hexagon on P2...), so It'll be fresh in my mind when doing the P1 VGA

Okay, am ready to make the VGA backend now. Should be done, like, tomorrow or the day after, hopefully

(Actually, wait no, I have something else to do in the weekend)

Well, I started messing with it and just remembered that 256px wide 8bpp VGA always needs that hub sync at 80MHz (forcing 20MHz pixel clock). UX Module runs at much higher clock so no sync may be fine? If not I fear the picture will have to be squished a bit. Will certainly have to develop on the module itself, which diminishes the usefulness in more general circumstances, but VGA on P1 sucks, anyways, so no big loss.

The existing text mode seems to work ok with odd 7.3MHz multiples. I've had to tweak timing things a little, but irrelevant really.
At the moment occupied fighting bit rot. Always on interface z180<~>Am9511A, who are not friends. Old binary works. New binary doesn't. Disassemble and line by line comparison produces no result, except self pity. 😖

Haven't really looked into what the text driver does. In theory this is actually really simple, just have to pump RDLONGs into WAITVID, but the bandwidth is so high.

Old binary works. New binary doesn't. Disassemble and line by line comparison produces no result, except self pity. confounded
I assume you don't have a working version in git? Because this sort of thing is where the git bisect tool really shines

I have a gist which I think matches. And a running old binary. And yet, I just can't recreate it working, and that is infuriating.

The relevant PASM blobs are identical, yet one binary doesn't work?
Have you thought about what gets loaded into RES'd / unused cog ram? That's usually a source of tricky problems.

Also, speaking of that text mode timing, have I ever mentioned that I can not get the clock/phase to line up on my particular monitor? Would be easier to get a grip on though if there was a way to fill the screen with a checkerboard

And yep, 256 wide 8bpp works fine at the UX module clock speed without funky code
ux_256widetest.zip

One thing that still annoys me is the screen flash that happens during booting the RC2014. It is almost like the Propeller is freezing for a moment.
But actually everything is happening on the RC2014 side. The UX Module is essentially just a slave at that point, so I don't have a reason for the screen flash.

Oh, my monitor doesn't like 640x480 85Hz very much, either. By which I mean it displays just fine, but has a big red "out of range" banner on top of it. IDK what that is about, the monitor really does suck.

triangles.zip
Got something going. Seems to glitch intermittently though, IDK what's going on.

graphtest.zip
Here's another example that feigns some amount of productivity and also shows off how terrible the line drawing function is.
Still glitches, will figure those out eventually.

Woo! Spinning Triangles! :tada:
Looks like you've already got it almost perfect!
Here are some proof videos from my end.
I can see the tearing on my monitor too. I captured a screenshot, which might give you a fingerprint to where it is coming from.

That's actually a really strange one. But I'm fairly certain it must have to do with the video out code, because the rendering code didn't really change from what is proven to work (well, I made the buffers relocatable...)

And yet, I just can't recreate it working, and that is infuriating.

Totally OT: I needed ld hl,(z_2) rather than ld (z_2),hl. Obvious in hindsight, and I also know how it happened. But now I can get my focus back here.

Oh, that is an infuriating one.

Figured out the glitches: there's a line in the VGA code that says wrbyte refreshCounter, syncIndicatorAddress. That should be commented out.

Here that is in action, also some more colors
triangles2.zip

Looks good. I made a youtube video to show it off for you.
It looks like the refreshCounter variable isn't used at all with that line deleted, so all references can be removed.
I'll need to learn how at least the render language works, so that we can design a good efficient way to interact with it from the Z80 side. I'll spend some time tonight looking into it, and will have some questions I'm sure.

Nice.
Oh BTW, I think all these examples have a bug: They forget to multiply DLIST_SIZE by 4 when passing it into gl.start, wasting ¾ of the memory...

No worries. Just check in what you're happy with and we can get going with the Z80 interface. It looks great as is.
I'm thinking of asking RC2014 Group for suggestions on the interface too. Always nice to be inclusive.

Yeah, will have to ASCIIfy it, clean some stuff up and then I'll check it in.

I'm thinking of asking RC2014 Group for suggestions on the interface too. Always nice to be inclusive.

Good idea, I think.

done

Also, if you're a lunatic who wants to do those kinds of things, you can flash this cut down version of Spin Hexagon to the UX module because why not
hexagon_uxm.zip

Wait, changing one of the color values for the stage from $EFEF to $ECEC (which is only changing the "don't care" bits) makes it have less glitches. I am very confused and will investigate if that is a general VJET bug. Tomorrow, that is
hexagon_uxm.zip

There's \draw in the demos that is a non affecting typo, right?
And the font file is needed too, right?
Will make a hexagon vid too if you think the demo is ready?

\method means that it catches any ABORTs that that method may throw. See propeller manual.
The font file is needed to draw text. You probably want a different one though. There is a font conversion helper script in the VJET 1.0 release ZIP.
You can try making a video of Spin Hexagon on the UX Module if you want and it doesn't glitch too much for you. It's really just here because it took like 15 minutes to port. Here's a build with a more accurate version string:
hexagon_uxm.zip

Ok. I didn't know about \ method. I will read about it.
I'll make another video, to explain why RC2014ers should be interested. Will help I'm sure.
Good luck investigating glitches 🤞(Daumen drücken).

I tried with the checked in versions, and I found the spinning triangles glitching horribly. I think the only thing you changed was the length of the display list. When I set it back to 900 the glitching disappeared.
Perhaps the display list length contributes to the glitching with Spin Hexagon?

I checked the files and they were fine. Will have to check tomorrow. Insufficient dlist space should not cause glitches, just missing shapes

There is a font conversion helper script in the VJET 1.0 release ZIP.

Sorry for being super doof, but I can't find where the font conversion helper is, nor where the "release" for VJET is either.
I've checked your Gitlab signature, and parallax forum, etc, but nothing looks right.

Just to add a bit to this glitching issue.
Experimenting with vjet_test.spin, value of DLIST_SIZE less than about 195 glitches for me.
Here's a 1 sec video of the glitch with DLIST_SIZE = 190

The files as committed (plus the font I forgot...) work perfectly fine for me and that is not the kind of problem insufficent dlist should cause.
Also yes, I thought that script was in there, but it is infact not. Here it is. Takes a PNG and spits out some character definitions that you need to manually edit a bit to get a useful font
font2spin.zip

Infact it isn't even close to using all the 150 longs. I can add a couple extra triangles and it's still fine on my end. What compiler do you use again?
vjet_test.zip

Using the Propeller IDE. Yes I can't imagine there being a direct issue. It may be in the VGA code rolling over or an addressing mode change. IDK really. But it is consistent and repeatable.

Works on PropellerIDE here, too.
Also, I think I may be a step closer to figuring it out: It happens a ton when I change it to use 4 render cogs. There was a similar issue with the P2 version. I think it has to do with the timing with which the video code updates the line counter.
vjet_test.zip

Can you look in the preferences for PropellerIDE and copy me whatever file is set as the compiler?

Ok. I'll have to dig around for the actual file, but in preferences it is set to bstc, and the version in the attached pic.
There is an openspin option for the spin compiler too. I haven't tried that, but I will now.

Ahhh

The preferences window in the version I have look totally different?
Mine was pre-set to openspin. They should all be equivalent. The "9 Constants folded" tells me BSTC's optimizations are enabled though, which could be affecting things.

Unrelatedly, I am observing that with that 4 cog file, the glitches mostly happen when the rainbow is pointing up. Is it running out of time for top edge fixup?

Hmm. Having a bit of trouble finding the actual binaries. And, trying to get openspin preference to upload to the Prop doesn't seem to work either. Perhaps an IDE bug.

Unrelatedly, I am observing that with that 4 cog file, the glitches mostly happen when the rainbow is pointing up.

I'm counting glitches with DLIST_SIZE = 150, and I don't think they are "up" related.
I see more glitching when the rainbow is on the diagonal. Either up-right or down-left are the most common (but not the only glitch time)  Up-right glitching causes a substantial frame sync issue too. Whereas other glitches are just momentary.

This is the "deb" that I used. All the files are just zipped up in here.
I can extract them if you prefer, but they should be easy to grab from here.
propelleride-0.38.5-amd64.deb.zip
I'm glad that you aren't seeing glitching because that rules out hardware.
Otherwise the Prop Forum would be sure it is related to 118MHz messing with video hardware.

Also I think the last file I uploaded was the wrong one

Anyways, does this binary here work for you?
vjet_test.zip

Will have to do tomorrow. 🛌💤

I just tried it with BSTC myself and I also can't reproduce anything there

I will try a real VGA screen tomorrow too.

I can tell you that that is not the issue.

oop wrong button

Interesting: moving the line buffers into high memory (normal VJET is hardcoded for this, I changed it for the UXM version because why not) changes something....
Seems to glitch a bit less, but still sometimes. Doesn't seem to have that connection to the rainbow position anymore? Eitherhow, this rules out out-of-bounds line buffer writes as an issue.
vjet_test.zip

vjet_test.zip
Managed to get it to glitch even more consistently.

Adding 200 longs of VAR (position wrt other variables doesn't seem to matter) makes it go away, apparently it is not a timing issue, but some sort of memory corruption.
vjet_test.zip

Also, messing some more with spin hexagon, I had it freeze on me after a video glitch (with intact display, oddly enough), so definitely memory corruption (there is no feedback from the graphics code that could cause this)

Here's an even more corrupted specimen - it just glitched for a couple seconds and then it stayed frozen on this

Yeah something's afoot.

Yeah ahh. I've played with the last three, and here's what I think is happening. There's a memory corruption issue related to overclocking the hardware. I've got several pieces of the UX Module hardware and they all behave very slightly differently with the same code.
Your last three tests with 4 render cogs have not produced a stable image for any of my Modules, although the third from last option (first version with 4 render cogs) was close. I don't have a 6.25MHz/5MHz crystal around, otherwise I could test the theory that it is hardware related.
For giggles, I've merged your new triangles with 3 render cogs, and it works fine albeit with the same slight glitch previously described (mauve and moving text).
Would it be possible to test that theory that there's a memory write fault (hardware related) by using a render "read" cog to verify by reading back the rendered? Or would it be a Schrodinger's situation where reading back would change the outcome?
Another wild thought is that one/some pieces of the VGA render code that were removed were needed to counter exactly this effect (possibly known issue) in the hardware? Perhaps there needs to be some separation between rendering and displaying?

Here's another wild guess. Updating FPorchStatus, VSyncStatus, and VSyncStatus hub memory in the tight video loop sometimes clashes with the render cogs all needing to write hub memory too, so the sync slips (glitch). Using 4 render cogs makes this situation worse (for me anyway), where the display cog may have to wait 23 x 4 cycles to continue, and worse case this could happen 3x per line.
Thoughts?

Have you tried any of the precompiled binaries I posted? Just to rule that all out?

Would it be possible to test that theory that there's a memory write fault (hardware related) by using a render "read" cog to verify by reading back the rendered? Or would it be a Schrodinger's situation where reading back would change the outcome?

I think you're thinking of Heisenberg's uncertainty principle, but I did suspect hardware corruption. This simple synthetic stress test I wrote didn't show any results for me though (if any of the bars change color, something went wrong...):
stresstest.zip

Another wild thought is that one/some pieces of the VGA render code that were removed were needed to counter exactly this effect (possibly known issue) in the hub memory hardware? Perhaps there needs to be some separation between rendering and displaying?

No.

Also this warning in the original code // For each included copy of this object only one spin interpreter should access it at a time. //. I guess that is for the PUB methods which may not be reentrant?

Idk why that was there, being an object that would have no issue being accessed in parallel. Irrelevant now because the only methods left are start and stop.

Here's another wild guess. Updating FPorchStatus, VSyncStatus, and VSyncStatus hub memory in the tight video loop sometimes clashes with the render cogs all needing to write hub memory too, so the sync slips (glitch). Using 4 render cogs makes this situation worse (for me anyway), where the display cog may have to wait 23 x 4 cycles to continue, and worse case this could happen 3x per line.

Well, it's not in the hot loop, it happens between lines. If there was a timing issue there, the monitor would loose sync.

Have you tried any of the precompiled binaries I posted? Just to rule that all out?

No, I've failed to upload any pre-binaries. I've now got the Windows Prop tool on a Windows system, and it worked once, but then no more. I was in the middle of rebooting (because Microsoft thinks I've nothing better to do), and I got distracted by your .spin code. I'll go back to try now.

This simple synthetic stress test I wrote didn't show any results for me though (if any of the bars change color, something went wrong...):

Bars don't change colour for me either. So that's something. Not hardware then.
Just thinking further about accessing hub memory in the video line loop. It should be deterministic and the number of render cogs don't matter. The wait will be 4 wrlong x23 cycles worst case every line, and that can't change, so just silly hypothesis. So no more guess at the moment.

Have you tried any of the precompiled binaries I posted? Just to rule that all out?

No, I've failed to upload any pre-binaries. I've now got the Windows Prop tool on a Windows system, and it worked once, but then no more. I was in the middle of rebooting (because Microsoft thinks I've nothing better to do), and I got distracted by your .spin code. I'll go back to try now.

Oh yeah, PropellerIDE doesn't have that as a feature. I think you should be able to do it on the command line with bstl or proploader

This simple synthetic stress test I wrote didn't show any results for me though (if any of the bars change color, something went wrong...):

Bars don't change colour for me either. So that's something. Not hardware then.

Or the test is just not hitting the problem....

Just thinking further about accessing hub memory in the video line loop. It should be deterministic and the number of render cogs don't matter. The wait will be 4 wrlong x23 cycles worst case every line, and that can't change, so just silly hypothesis. So no more guess at the moment.

Again, the video timing is fine, the monitor would loose sync if it wasn't.

I think you should be able to do it on the command line with bstl.

I've uploaded this binary with bstl (which correctly found my UX Prop even though the Parallax tool says it is not connected, go figure). I've sent a link for video of the outcome to email address.
I'm seeing loss of frame sync.

Yeah, that one is perfectly stable for me. Owie ouch, something bad's going on.
To get the propeller tool to see your prop, you need to set the reset signal to "DTR & RTS" in the preferences. There you can also tell it which COM port to use so it doesn't have to probe all of them every time.

I've been playing around with the VGA display engine, trying to understand it.
And frankly, I don't understand how it is working.
I thought that frqa should be a simple count of the number of cycles per pixel, which is set by ctra to be clkfreq / 4 in our case.
But rather than just calculate the correct frqa using clkfreq/4 directly, the code does a strange division-like calculation.
And your "seemingly arbitrary" initial value of 20_000_000/2 gets the correct answer, whereas the original calculation using a known quantity of 25_175_000 + 1_600) / 4 is wrong. The original calculation looks a bit suspect with a small offset added to though, BTW.
I was playing with the theory that the VCO is going out of range, and that perhaps dividing the VCO by 8 would help. Just a theory though.

Yes, FRQA is the frequency, but it's obviously not in hertz, but in units of clkfreq/4294967296. Also, the phsa overflow doesn't directly control the pixel generation. It first goes into a PLL where the frequency is multiplied by 16 (also helps reduce jitter) , then through a simple 2^n divider and then finally into the video hardware. The video hardware then derives the pixel clock by a further division (by the bottom part of the VSCL register)
The 20_000_000 is not arbitrary, it's just the pixel clock for 512x480.
TBH idk what that code does but it sets the right frequency, so eh.
As said above, I don't think the VGA out code is the problem. If it was the problem would look different. And that doesn't explain the memory corruption that's going on.

I guess the other approach is to write a parser integrated with the Z80 interface code, and use that to generate calls to your display list helper code. There are only a few calls to recognise so it wouldn't be too hard.
I was just concerned by performance of having Spin in the critical path. But, perhaps that's not a valid concern?

Spin is actually not that slow compared to an 8 bit CPU. In particular, the multiply/divide routines are at native speed. Would need some benchmarking though.
An easy way out would be to additionally give the Z80 a way to write shape bodies directly to the list, bypassing the mathy stuff. Useful for static "sprites" and stuff, too.

Have you figured out the weird corruption we were seeing yet?
Because I've got frustrated with it and started doing something else. (sorry).

Not yet. But I'm still not convinced that it's not an overclock glitch.
Sometimes, these things just need to stew for a while, and the solution becomes obvious.
I've been working on VT100 support and next will be the revised z80 interface.
Need to put some thoughts in the interface design first though. So actual code progress may be slow.
When I have questions about how things work, I'll ping you for help.

Thinking that the obvious "standard" to work to is GSX-80, which should enable a small initial application portfolio.
The VDI Specification is defined in Appendix B of the GSX-80 Graphics Extension Programmers Guide.
This document describes the GDOS calls to which a GIOS driver must respond, and the the contents of the calls. So, on the Z80 side the driver is simply pushing Bytes. The magic will be on the Propeller side in Spin so ideally simple to maintain.

Should be possible. If it works for ol' vector terminals, VJET should also work as a backend.

Still ruminating on the Z80 interface. Mainly how much alignment to the VDI is sensible for the actual Z80 bus interface. The VDI  has pointers to pointers, which don't translate to an definable transaction.  So a pure VDI definition doesn't seem possible. Also on the Z80 driver side there will need to be some translation of easy VDI commands that don't require hardware access to fulfil.
Just playing randomly with the demo again makes more evidence that the corruption is memory position orientated. Increasing the DL to as large as it can go (about 3000) makes the glitching consistent, yet commenting out the background triangle (with outside display window dimensions) mostly recovers the screen to just a few glitches. Curious. No closer to resolution though.

(sorry for replying late, didn't see the question first time I read this)

Changing the dimensions of the background triangle to keep it "onscreen" causes no improvement, but changing the colour from $ECEC to (for example) $1010 does make things more stable. Is there something special about how colour is written or managed? Hmm.
Yeah, I've noticed that particular colors cause the issue. VJET does not do anything with the color but write it to RAM. I don't think that if it were overwriting something important, it would matter what it is writing, so either writing certain bytes to ram causes hardware glitches or it has something to do with these bit patterns appearing on the VGA output (power consumption?)


grains:total impacted by rounding errors

Becuase total is a float it is actually comparing:
`Error: Test failed: 'returns the total number of square on the board'

total() not equal to 1.844674e+19.  If you use option(digits = 22) you can see:* total() not equal to<PHONE_NUMBER>3709551616.`

Test was supposed to be:
  expect_equal(total(),<PHONE_NUMBER>3709551615)
See thread for more discussion.  Do you want to teach bigInts and importing libraries this early in the track?

Hello @gamecock,
could you please post your sessionInfo() output? I'm not sure why you are seeing this different behavior. Here's mine:
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] testthat_2.0.0

loaded via a namespace (and not attached):
[1] compiler_3.5.0 magrittr_1.5   R6_2.2.2       tools_3.5.0    withr_2.1.2
[6] yaml_2.2.0     rlang_0.2.2

@jonmcalder Can you reproduce the test fail that Mike reports?

sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
loaded via a namespace (and not attached):
[1] compiler_3.4.3 tools_3.4.3    yaml_2.1.18

Thank you. The easiest possible culprit to exclude is your R v3.4.3 (although there should have been other users noticing this problem then). Can you try upgrading to 3.5.x from one of the mirrors?

sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
loaded via a namespace (and not attached):
[1] compiler_3.5.1 tools_3.5.1

Same behavior, this should fail.
library(testthat)

expect_equal(18446744073709551616,<PHONE_NUMBER>3709551615)

The solution is:
library("gmp")
library("testthat")
expect_equal(as.bigz(2)^64, as.bigz(2)^64-1)
Error: as.bigz(2)^64 not equal to as.bigz(2)^64 - 1.
'target'(bigz) and 'current' differ

… isn't the expected result that if you try to compare two large numbers beyond the size of an Integer you lose precision?

Ah, no I see what you mean. identical(18446744073709551616,<PHONE_NUMBER>3709551615) yields TRUE here, as well, but it shouldn't. Sorry for the version detour!
Which "thread for more discussion" do you mean?
@jonmcalder, @jkanche & @amoradell: What do you think about teach[ing] bigInts and importing libraries this early in the track?

Sorry for the delayed response - I'm travelling at the moment so haven't got much capacity for keeping tabs on issues here.
Am I correct in saying that the issue @Gamecock is raising here is the same as referred to in #84?
Essentially base R doesn't provide support for large integers and we didn't have consensus previously on how to deal with this.
My sense (revisiting it again now), is that the issue of precision is in some ways fundamental to the problem and we should have a way of supporting more precise testing for the exercise.
Two potential solutions come to mind:

include a hint to use a particular R package to provide support for big integers and extend the exercise with optional tests (i.e. commented out by default) that implement more precise matching using the relevant type from this package (making this more advanced solution optional should then mean we can keep this exercise early in the track)
alternatively we could move the exercise later in the track, have a hard requirement to use a specific library and include the relevant test as a "required" test to pass for the exercise

@jonmcalder that is the correct issue, I didn't think to look through closed issues.  I have sufficient skills to do the work, but don't have enough experience with r to determine the correct path.

+1 for the 2nd option.

Ok cool. I'm happy with the second option too. @gamecock are you happy to get started on a PR and then raise any further questions on implementation details with @katrinleinweber or me?

Yes,
 I'll update the exercise, and then seek your input about where in the

track to place it.
On Fri, Sep 7, 2018 at 2:03 AM Jon Calder<EMAIL_ADDRESS>wrote:

Ok cool. I'm happy with the second option too. @Gamecock
https://github.com/Gamecock are you happy to get started on a PR and
then raise any further questions on implementation details with
@katrinleinweber https://github.com/katrinleinweber or me?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/exercism/r/issues/106#issuecomment-419332418, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFkhSp0m5gcCXvobl23IWCa1a8kYZQnhks5uYgwagaJpZM4WC89b
.

I approve second option.
I suppose that core exercices and easy ones should be made with R base package.

Hi, I'm not sure if I understand the above discussion, or if it is even referring to the same issue but I saw  the first post said

Becuase total is a float it is actually comparing:
`Error: Test failed: 'returns the total number of square on the board'
total() not equal to 1.844674e+19. If you use option(digits = 22) you can see:* total() not equal to<PHONE_NUMBER>3709551616.`
Test was supposed to be:
expect_equal(total(),<PHONE_NUMBER>3709551615)

but I found that for the test
test('total', () => {
    expect(grains.total()).toBe('18446744073709551615');
  });

using big-integer.js gave me an answer of '18446744073709551616' no matter what I did.  So I looked at other people's answers to figure out what I was doing wrong.
What I found was that other people were just subtracting 1 from the 'total' method in their answers using a variety of different fixes so I did the same.  There was only one test involving the total method so all tests passed that way.
However, upon reflection, maybe the test should be fixed.
For sake of completeness, here was my total method:
total() {
    let tot = bigInt(0);
    for (let i = 2; i < 65; i++) {
      tot = tot + parseInt(this.square(i))
      }
      return bigInt(tot).minus(1).toString();
    }

The .minus(1) in the second to last line was my "fix".
Correct me if I'm wrong, but it sounds like the test used to test for the number I and everyone else I looked at was getting?  That's sort of what I got from @Gamecock said

total() not equal to 1.844674e+19. If you use option(digits = 22) you can see:* total() not equal to<PHONE_NUMBER>3709551616.`
Test was supposed to be:
expect_equal(total(),<PHONE_NUMBER>3709551615)

Or,  if I really am supposed to get '18446744073709551615' what should I be doing instead?
My full answer is here: https://exercism.io/my/solutions/2261a8c46c844d43992a4ac2c7ba5e03

Hi @beansprout! We're the R track here ;-) Please have a look at exercism/javascript/search?q=grains & maybe move your issue / comment there. Cheers!

Oops! my bad @katrinleinweber - opened new issue.  Thanks!


Add support for Bean Validation 2.0

This seems like a pretty heavy process. While the validation will be run on the server it does not tell anything about the required validation to the clients besides the documentation. Maybe this should be exposed as directives instead?


Run all evals as jobs within a single Beaker experiment

I find it convenient to have all evals for a given model associated with a single experiment.
This PR implements this. It also has the added side effect only creating 1 new template yaml instead of n_tasks of them. I'll comment the code below to explain what's happening.
I've tested this (for instance the beaker job above) and it seems to work. This is also purely personal preference, so if you don't like this we can just make it a separate script called submit_eval_experiment or something; whatever you guys prefer.

You can cancel jobs independently. To do this, you click on the job you way to cancel, copy the hash after the last slash in the URL (that's the job id), and do beaker job cancel [job-id].
You can only restart at the experiment level rather than job level. By default, when you call beaker experiment resume [experiment-id], Beaker will check for any failed or stopped jobs in the experiment and re-run them, which seems reasonable? We could ask the beaker team to add functionality to restart individual jobs if you think this is important though.

Do you mean that if you hit restart, it only restarts jobs that have failed or were cancelled? That's fine for now, just making sure that you don't rerun everything if one task fails for an ephemeral reason (e.g., RAM issues).

Correct, it only restarts failed or canceled jobs and leaves the completed ones untouched.


WhereILike - Fix typo in types

Fixing typo:  whereIlike to whereILike in types.
Changes introduced in https://github.com/knex/knex/pull/4779

Released in 1.0.2


feat: automatically convert raw nested data to structures

Codecov Report
Attention: Patch coverage is 95.23810% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 90.28%. Comparing base (fd62d80) to head (376b085).
Report is 1 commits behind head on main.

Files
Patch %
Lines

src/Structs/AbstractStruct.php
94.44%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@              Coverage Diff              @@
##               main       #4       +/-   ##
=============================================
+ Coverage     62.41%   90.28%   +27.86%
- Complexity      118      126        +8
=============================================
  Files            12       12
  Lines           306      319       +13
=============================================
+ Hits            191      288       +97
+ Misses          115       31       -84

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Syntax Coloring Latency

Hello,
I am not sure how to explain this issue, but I notice significant overhead when using Syntax Coloring, specifically, frequently when I am typing, the VS text editor 'freezes' input for roughly 1 to 2 seconds or so, my suspicion is that the cause is the Syntax Coloring (or something related to it) as when I disable it, then close the text editor and reopen it, the problem seems to go away.

What profiler says?

Hi @vasily-kirichenko , I produced the behaviour and copied the output window:
[VFPT][Information 09:23:01 AM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.3352192
getCategoriesAndLocations, 00:00:00.3142426
GetAllUsesOfAllSymbolsInFile, 00:00:00.0194682
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.0194625
LS allSymbolsUses, 00:00:00.0155502
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.0026046
ParseFileInProject, 00:00:00.0014041
LS ParseAndCheckFileInProject, 00:00:00.0012935
[VFPT][Information 09:26:29 AM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.1517972
ParseFileInProject, 00:00:00.1004051
GetAllUsesOfAllSymbolsInFile, 00:00:00.0256742
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.0256612
getCategoriesAndLocations, 00:00:00.0256038
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.0212259
LS allSymbolsUses, 00:00:00.0022332
LS ParseAndCheckFileInProject, 00:00:00.0021929
I don't think these numbers represent the delay I am seeing?

You are right.
@dungpa It seems we call FCS by Async.StartImmediately. I tried to switch to Async.Start everywhere and it was crashing constantly. We should carefully examine why StartImmediately was used initially and eliminate all the usages where possible.

I think certain operations require to run on UI thread, but some uses of StartImmediateSafe are superficial. You can try to eliminate those constantly running on cursor changes first.

@vasily-kirichenko What I meant is to change StartImmediateSafe to StartInThreadPoolSafe, identify some code fragments that access UI components and switch back to UI thread explicitly (see https://github.com/fsprojects/VisualFSharpPowerTools/blob/bc76dc272cb941fdbc04e2a0d203a685c5d3bc84/src/FSharpVSPowerTools.Logic/GoToDefinitionFilter.fs#L406-L451). It's a bit of pain, but it's doable if we would like to move most of the things off the UI thread.

@xenocons We tried to address this issue via #975. Really appreciate if you can spend some time to test this beta build https://ci.appveyor.com/project/dungpa/visualfsharppowertools/build/1.9.0-a1505251346/artifacts. If there is still UI freeze, please let us know. Thanks.

Hi @dungpa , i have been testing with the beta build and VS2015 RC and i still get stuff like this:
[VFPT][Information 08:55:20 AM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:01.5645482
ParseFileInProject, 00:00:00.9148228
GetAllUsesOfAllSymbolsInFile, 00:00:00.5820789
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.5820538
LS ParseAndCheckFileInProject, 00:00:00.3055008
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.2605910
getCategoriesAndLocations, 00:00:00.0674381
LS allSymbolsUses, 00:00:00.0159509

Quite consistently it seems to be that 'GetAllUsesOfAllSymbolsInFile' takes the largest amount of time across all debug outputs, not uncommon to see it taking 150~400 msec

@xenocons The idea of our fix is to move heavy computations out of UI (main) thread whatever possible. GetAllUsesOfAllSymbolsInFile, instance.GetAllUsesOfAllSymbolsInFile and ParseFileInProject are all calls to the F# compiler wrapped as F# Compiler Service (FCS) library (NuGet package) which we use for analyzing code. We cannot make FCS faster, we only try to remove any UI freezes (which typically occur during active code editing). So, on large files parsing and checking will always take several seconds, or even a minute or two.

hmm ok, what about getCategoriesAndLocations, they both seem to be the next candidates (see below, ignoring the exception which doesn't seem related to perf issues?). We might be splitting hairs here though. FYI the files I have in my project are not really what I would call large, they average about 400 LoC, and there is about 8k LoC in the solution.
[VFPT][Error 09:21:54 PM] Exception Message: Index and length must refer to a location within the string.
Parameter name: length
Stack Trace:    at System.String.Substring(Int32 startIndex, Int32 length)
   at FSharpVSPowerTools.StringCategorizers.lines@89.Invoke(Int32 line)
   at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc`2 mapping, FSharpList`1 x)
   at FSharpVSPowerTools.StringCategorizers.categorize[a](Category category, Regex regex, FSharpFunc`2 getTextLine, range r)
   at FSharpVSPowerTools.StringCategorizers.EscapedChars.getCategories@134.Invoke(range r)
   at Microsoft.FSharp.Primitives.Basics.List.mapToFreshConsTail[a,b](FSharpList`1 cons, FSharpFunc`2 f, FSharpList`1 x)
   at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc`2 mapping, FSharpList`1 x)
   at Microsoft.FSharp.Collections.ListModule.Map[T,TResult](FSharpFunc`2 mapping, FSharpList`1 list)
   at FSharpVSPowerTools.StringCategorizers.EscapedChars.getCategories[a](FSharpOption`1 ast, FSharpFunc`2 getTextLine)
   at FSharpVSPowerTools.SourceCodeClassifier.getCategoriesAndLocations[a](SymbolUse[] allSymbolsUses, ParseAndCheckResults checkResults, LexerBase lexer, FSharpFunc`2 getTextLine, FSharpList`1 openDeclarations, FSharpOption`1 allEntities)
   at <StartupCode$FSharpVSPowerTools-Logic>.$SyntaxConstructClassifier.spans@264-2.Invoke(Unit _arg12)
   at FSharpVSPowerTools.Profiler.Time[b](String msg, FSharpFunc`2 f)
   at <StartupCode$FSharpVSPowerTools-Logic>.$SyntaxConstructClassifier.worker@262-31.Invoke(SymbolUse[] _arg12)
   at Microsoft.FSharp.Control.AsyncBuilderImpl.args@835-1.Invoke(a a)
[VFPT][Information 09:21:55 PM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.4116379
ParseFileInProject, 00:00:00.1758745
getCategoriesAndLocations, 00:00:00.1353445
GetAllUsesOfAllSymbolsInFile, 00:00:00.0748361
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.0748171
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.0694045
LS allSymbolsUses, 00:00:00.0036427
LS ParseAndCheckFileInProject, 00:00:00.0017599

[VFPT][Information 09:22:17 PM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.5803634
ParseFileInProject, 00:00:00.3422934
getCategoriesAndLocations, 00:00:00.1388409
GetAllUsesOfAllSymbolsInFile, 00:00:00.0990271
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.0990028
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.0604703
LS allSymbolsUses, 00:00:00.0361969
LS ParseAndCheckFileInProject, 00:00:00.0023229

[VFPT][Information 09:22:43 PM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.5764450
ParseFileInProject, 00:00:00.3507992
getCategoriesAndLocations, 00:00:00.1206598
GetAllUsesOfAllSymbolsInFile, 00:00:00.1047948
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.1047765
LS ParseAndCheckFileInProject, 00:00:00.0621087
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.0327221
LS allSymbolsUses, 00:00:00.0099350

[VFPT][Information 09:22:50 PM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.2958585
ParseFileInProject, 00:00:00.1228455
getCategoriesAndLocations, 00:00:00.0959328
GetAllUsesOfAllSymbolsInFile, 00:00:00.0768881
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.0768702
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.0650581
LS allSymbolsUses, 00:00:00.0101231
LS ParseAndCheckFileInProject, 00:00:00.0016797

[VFPT][Warning 09:25:01 PM] Can't read Visual Studio themes from registry keys.
[VFPT][Information 09:25:11 PM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.2993082
GetAllUsesOfAllSymbolsInFile, 00:00:00.1809116
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.1808918
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.1737386
getCategoriesAndLocations, 00:00:00.0783531
ParseFileInProject, 00:00:00.0398567
LS allSymbolsUses, 00:00:00.0037438
LS ParseAndCheckFileInProject, 00:00:00.0033987

[VFPT][Information 09:25:17 PM] [SyntaxConstructClassifier] [Fast stage]
Total = 00:00:00.7607649
ParseFileInProject, 00:00:00.3787960
GetAllUsesOfAllSymbolsInFile, 00:00:00.2833059
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.2832865
LS GetAllUsesOfAllSymbolsInFile, 00:00:00.1560695
LS ParseAndCheckFileInProject, 00:00:00.1117131
getCategoriesAndLocations, 00:00:00.0984307
LS allSymbolsUses, 00:00:00.0154943

Another one, 'getOpenDeclarations', i have witnessed it taking a second:
[VFPT][Information 12:00:13 PM] [SyntaxConstructClassifier] [Slow stage]
Total = 00:00:01.9359468
getOpenDeclarations, 00:00:00.9238624
GetAllUsesOfAllSymbolsInFile, 00:00:00.6248944
instance.GetAllUsesOfAllSymbolsInFile, 00:00:00.6248625


[v2] [tiptap warn]: Duplicate extension names found: ['bubbleMenu']. This can lead to issues.

Hi, i use tiptap field multiple times on the page, and this what i have in console in chrome

Yep. This is a bug in Tiptap. I'm hoping they solve it soon.

In my opinion this is a breaking change as it completely removed source code I put into code blocks, so that the next time I save in Filament PHP everything is gone. Do you have any clues or insight why this happens, e.g. the bug report?

The paragraph button is hard coded because there should always be a way to reset anything to the default state. So it alone doesn't honor the profiles or the tools.


[UI] I want to add a default Avatar for who added this campsight.

Description
For now in 'view details' page the owner of this list is not visible in flash.  It mixed with other details and looks like normal as usual. Which is not good for User and also who added this.
Current Behavior
Currently it's look like this

Proposed Behavior
But I want to make it like this, Which is more visually appealing

Impact
It's directly impact in UX of this platform because displaying owner along with it's name is more convenient for user and also very appealing.
Please assign me this issue I want to implement this Feature. Thank you.

Hey @Suvadip-sana , You can proceed!


Aggregate counts when local outlier flags are to be excluded from analysis

Need to add a function to remove/exclude flagged local outliers counts for each segment in the aggregate counts function.

Added to setbioprobeqc function option to remove locals; i.e. change to NA in exprs so that when aggregate occurs these values are ignored.


Avoid modifications to the informer's copy of resources.

In each of the {Pipeline,Task}Run reconcilers the functions to update status and labels/annotations refetch the resource from the informer cache, check the field they want to update, and if an update is needed they set the field on the informer's copy and call the appropriate update method.
In pseudo-code:
func update(fr *FooRun) {
  newFr := lister.Get(fr.Name)

  if reflect.DeepEqual(newFr.Field, fr.Field) {
    newFr.Field = fr.Field   // This modified the informer's copy!
    return client.Update(newFr)
  }
}

Changes
I have worked around this in two different ways:

For the status updates I added a line like newFr = newFr.DeepCopy() immediately above the mutation to avoid writing to the informer's copy.

For the label/annotation updates, I changed the Update call to a Patch that bypasses optimistic concurrency checks.  This last bit is important because otherwise the update above will lead to the first reconciliation always failing due to resourceVersion skew caused by the status update.  This also works around some fun interactions with the test code (see fixed issue).

There are two other notable aspects to this change:

Test bugs! There were a good number of places that were assuming that the object stored in the informer was altered.  I changed most of these to refetch through the client.
D-Fence! I added some logic to some of the common test setup code to DeepCopy() resources before feeding them to the fake clients to try and avoid assumptions about "same object" creeping back in.

It is also worth calling out that this change will very likely destabilize the metric that I identified here as racy, which is likely masked by the mutation of the informer copies.
Fixes: https://github.com/tektoncd/pipeline/issues/2734
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[ x ] Includes tests (if functionality changed/added)
[ n/a ] Includes docs (if user facing)
[ x ] Commit messages follow commit message best practices

See the contribution guide for more details.
Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
N/A
/hold
I'm throwing a hold on this due to the lingering t.Skip / DO NOT SUBMIT, which I am asking about in slack, but I wanted to fling this at CI to see what falls out.

/kind bug

This is great, thank you!
I started looking into it, it's not clear to me yet why the TestReconcileWithPipelineResults - whether it's a problem of the test or if it's a problem of the reconciler code.
If the former I think it might be ok to merge this, and go fix the test afterwards.

/retest

/retest

Once we start fetching specs from OCI containers, we'll need an equivalent of the informers cache for that too!

/hold cancel

/test pull-tekton-pipeline-build-tests


Change in Docker-Airflow local executor's Volume

The missing double quotes in local-runner's volume prevent users from accessing respective, virtual directories.
Issue #, if available:
Description of changes:
The missing double quotes in local-runner's volume prevent users from accessing respective, virtual directories.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Please update PR to reflect current 2.2.2 version


Components without a key in an array break whole app

Description
I have a component with the following render function.
  render() {
    return (
      <View style={{ flex: 1 }}>
        {this.props.children}
        {this.props.barEmbedded
          ? [
              this.props.renderBar(this.pullUpResponder),
              this.renderExpandedContentIfNeeded()
            ]
          : null}
      </View>
    );
  }

If the component returned by renderBar doesn't have key specified nothing is rendered (i.e. screen is blank) and there's no error/warning. The moment I add key to the component everything behaves correctly.
Reproduction
I tried reducing it but couldn't reproduce it in a small example.
Solution
No idea, sorry.
Additional Information

React Native version: 0.43.0-rc.0
Platform: iOS
Operating System: macOS
Dev tools: Xcode 8.2.1, iOS 10.2

Does this happen on 0.42?

No, I just checked. In 0.42 the element is rendered and there's a warning as expected.

Ooops, it's gone in 0.43.0-rc.2. Very interesting. I also moved from React 16.0.0-alpha.3 to alpha 4.


Formally specify correctness property of the chain sync mini-protocol with fixed chain

We shall formally specify the equivalence between sending a whole chain and running the chain synchronization mini-protocol when the client starts from the genesis block and the server is given the aforementioned chain, which then is never updated during the protocol run.
Additionally, since the definition of the semantics of mini-protocol programs (see #96) will likely not be ready when this GitHub issue is addressed, we shall assume the existence of the following semantics function:
⟦_⟧ :: ('p ⇒ 'm or_done program) ⇒ process family

I think that it we should use either program families and process families or programs and processes. I guess that the latter is enough. To go to process families we could start with program families and apply ⟦_⟧ pointwise.

I think that it we should use either program families and process families or programs and processes. I guess that the latter is enough.

My current code attempt (which follows your former approach) is the following:
consts protocol_semantics :: "('p ⇒ 'm or_done program) ⇒ process family" (‹⟦_⟧›)

consts sync_repeated_send :: "'a sync_channel ⇒ 'a ⇒ process family" (infix ‹◃⇧∞⇘s⇙› 52)

definition list_sender :: "'a::embeddable sync_channel ⇒ 'a list ⇒ process family" where
  [simp]: "list_sender c xs = foldr (λx p. c ◃⇘s⇙ x; p) xs 𝟬"

context chain_sync
begin

definition spec :: "'i list ⇒ process family" where
  [simp]: "spec C = list_sender client_chains [C'. C' ← prefixes C, C' ≠ []]"

 definition impl :: "'i list ⇒ process family" where
  [simp]: "impl C = ⟦program⟧ ∥ server_chains ◃⇧∞⇘s⇙ C"

theorem fixed_chain_sync_from_genesis_conformance:
  assumes "initial_client_chain = [hd C]"
  shows "spec C ≈⇩s impl C"
  sorry

end

To go to process families we could start with program families and apply ⟦_⟧ pointwise.

Do you suggest to do something like ⟦program Client⟧ ∥ ⟦program Server⟧? If so, how could I turn this compound process into a process family?


Add system.linux.memory.available metric

Fixes #257
Changes
Add definition for a single metric system.linux.memory.available.
Merge requirement checklist

[yes ] CONTRIBUTING.md guidelines followed.
[yes] CHANGELOG.md updated for non-trivial changes.
[not needed] schema-next.yaml updated with changes to existing conventions.

That will need a rebase to resolve the conflict in the changelog. @ItsLastDay could you please rebase?

Sure, done!

How do I merge this PR? Github says "The base branch restricts merging to authorized users. Learn more about protected branches.". I don't understand what I am missing :\

How do I merge this PR? Github says "The base branch restricts merging to authorized users. Learn more about protected branches.". I don't understand what I am missing :\

Only @open-telemetry/specs-semconv-maintainers can merge


Remove max version constraint on activemodel

User Service uses this gem, and I want to upgrade it to newer Rails version...

I'm not positive this will work with ActiveModel 5.0. Have you tested it out with this branch in user_service?

We might have to make a 5.0 branch of this validator.

I haven't tested it yet, no.  Will do...

I've tagged this as 2.3.0-rails5 for testing.


Change HQ Raleigh to Gold level sponsorship

done

need to update the partners poster as well

poster is drafted and ready for printing https://drive.google.com/open?id=1yP_SZWIsvVy0i8UXduXKqYB_-WjZk4uB


Instagram on iOS: the Instagram share option doesn't appear in the standard iOS menu when the message option is passed to open()

I tried installing both the current release and what's on the master branch. Both are producing this issue:
      await RNShare.open({
        url: `data:image/jpeg;base64,${base64data}`,
        type: "image/jpeg",
        message: 'test share message',
      });

When you pass a message option, the icon to share to Instagram disappears from the standard iOS share menu. If you remove the message option, the Instagram option comes back and you can share the image successfully.
Instagram works fine in both scenarios on Android.

Also facing the same issue, although this may be working as expected because when you post a picture to Instagram, you first edit the picture then two screens later you provide a message.

Right, similar to Facebook share dialogs, I expect Instagram may not allow "default messages" being passed via an SDK. However, the Facebook share option degrades gracefully by ignoring the message option but still attaching an image (in my testing). It would be great if the Instagram option did the same instead of the icon disappearing entirely. 😞

@tribou do you know any workaround on this?
maybe we can improve react-native-share

@erikterwiel yes its working as expected. Instagram doesn't accept the string message. So when we pass string as params, it won't show up in the share pop-up.
As a work around, we can copy the message to clipboard and pass only image as a params to instagram. Later we can paste the string.

something like this
Share.open(shareImageBase64);
const strToCopy = this.state.selectedQuote
.concat("\n.\n")
.concat(this.generateHashTags());
Clipboard.setString(strToCopy);
if (Platform.OS === "android") {
ToastAndroid.show(
"Caption along with hashtag has been copied.",
ToastAndroid.SHORT
);
} else if (Platform.OS === "ios") {
AlertIOS.alert("Caption along with hashtag has been copied.");
}
const shareImageBase64 = {
title: "Auto Captions",
url: data:image/png;base64,${this.img.data}
};

@jgcmarins no workaround yet, but I agree that it would improve react-native-share to degrade gracefully by default on iOS (like it does on Android already).

Nice Workaround:
React Native has a built in { Share } feature which works as expected on iOS, degrades gracefully on Facebook and Instagram. Only issue with the built in share is that you cannot supply a url for Android, so I just use Platform.OS to use React Native's Share for iOS and react-native-share for Android.
Works perfectly

Well, this is strange. I've tried your workaround @erikterwiel, and it's still not working. 😞 Very odd and slightly unsettling that this would be environmental. I can pass a title and/or a subject, but as soon as I add a message, the Instagram icon disappears using React Native's own { Share } module.

@tribou I have it working on react-native 0.56 and my message is ~200 characters

Ahhh, I'm still on v0.55.4. I had tried the 0.56 upgrade, but Babel 7 broke my test suite. I'll have to find some time to redo to just test this workaround... thanks!

Posting here since it's the next step I can think of for this issue: Compare<EMAIL_ADDRESS>iOS share module/ActionSheetManager with react-native-share and<EMAIL_ADDRESS>to see what changed...

`react-native@v0.56.0:/Libraries/ActionSheetIOS/RCTActionSheetManager.m
`react-native@v0.55.4:/Libraries/ActionSheetIOS/RCTActionSheetManager.m
comparison of the two react-native versions
current react-native-share iOS implementation of open()

@Runaider I'm also having a problem with 0.57

same issue in "react-native": "^0.57.8"

We did a new release, can you try it? v1.2.0.
Thank you.

Same issue...

Same issue. Do you have any solution?

No, please re-open this issue.

Re-open this issue please!

Does anyone have any reproducible bug?
Does it happens with master code?


nukeReferences: fix for store paths with special characters

Motivation for this change
I'm running nix with store directory in a custom path containing a '+' character. This prevents nukeReferences from working properly because some characters in the path get interpreted as a regular expression special characters. Broken nukeReferences will prevent python3 from building on my setup because it requires disallowedReferences = [ openssl.dev ];.
The proposed fix implements escaping of the special characters in the store path.
Things done
Tested with
with import <nixpkgs> {};

stdenvNoCC.mkDerivation {
  name = "nuke-refs-test";

  nativeBuildInputs = [ nukeReferences ];

  buildCommand = ''
    cat > $out <<EOF
    out: $out
    openssl: ${openssl}
    python: ${python}
    EOF
    nuke-refs -e ${openssl} $out
    echo
    echo "> Only out and python should be nuked:"
    cat "$out"

    echo
    echo "> This should fail:"
    nuke-refs -e /not/a/path $out
  '';
}

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@volth Could you please look at this? I think this is correct, but I don't have much experience with Perl.

LGTM
I wonder that it is the only problem with special characters in nix store path (I was tried to put nix store under C:\Program Files\ on Windows, but quickly gave up)


Puzzles about images shown in "Terminology" section in README.adoc

Dear authors,
I have some questions about the images used to explain the terminology.
Q1.  In the image "The following image shows how Span and Trace look in a system"， are "Span Id=C“,"Span Id=E“ and "Span Id=G“ all physical spans ? Would they be reported to zipkin ?
In my opinion，"Span Id=C" is equivalent to part of “Span Id=B” (Server Received and Server Sent in Span Id=B)， the same as  "Span Id=E" and “"Span Id=G”。And in the following explaination of "Why is there a difference between the seven and four spans in this case"， it seems that "Span Id=C“,"Span Id=E“ and "Span Id=G“ are referred to "Server Received (sr) and Server Sent (ss) events took place on the service2/3/4 side 。
Q2. In the image "The following image shows how parent-child relationships of spans look",  it shows that the "Span Id=C“ is a physical span and the parent span of “Span Id=D” and "Span Id=F”, is that corresponding to the database storage of zipkin ( that's the datatable of  zipkin_spans in zipkinv2  ) ?
I have tried out a demo to show the trace info just like the live demo in PCF, but the trace data stored in zipkin_spans  seems that  the parent span of “Span Id=D” and "Span Id=F” is "Span Id=B“.  Does that mean "Span Id=C“ doesn't exist in the implementation ?
Maybe I misunderstand some concepts. Look forwar to your reply.

are "Span Id=C“,"Span Id=E“ and "Span Id=G“ all physical spans ? Would they be reported to zipkin ?

yes

n my opinion，"Span Id=C" is equivalent to part of “Span Id=B” (Server Received and Server Sent in Span Id=B)， the same as "Span Id=E" and “"Span Id=G”

Ok, but that's not the case. We've simulated that someone has created a new span by themselves (maybe I need to update the docs to emphasise that).

Q2. In the image "The following image shows how parent-child relationships of spans look", it shows that the "Span Id=C“ is a physical span and the parent span of “Span Id=D” and "Span Id=F”, is that corresponding to the database storage of zipkin ( that's the datatable of zipkin_spans in zipkinv2 ) ?
I have tried out a demo to show the trace info just like the live demo in PCF, but the trace data stored in zipkin_spans seems that the parent span of “Span Id=D” and "Span Id=F” is "Span Id=B“. Does that mean "Span Id=C“ doesn't exist in the implementation ?

I don't understand the problem. The parent-child relationship is pretty straightforward. How it's stored in the Zipkin's DB is an impl detail I guess. If you're interested you can ask that question in the Zipkin repo.


How to automatically play video + audio in a page

If there is a video file with sound on the page, it cannot be played automatically due to the limitations of Chrome 66. However, when I tested the electron packaged page, it can be played automatically. Can we use photino to achieve this effect by setting parameters?
Chrome 66 autopaly
https://developer.chrome.com/blog/autoplay?hl=zh-cn

@whgfu We use WebView2 as the browser control on Windows, WebKitGTK on Linux and WKWebView on Mac. We do support passing startup parameters to each browser control, so if the browser controls support this feature via a switch, you can pass it. From that point, it's up to the what the control supports. Some of the more common switches are supported with custom parameters, but each also supports passing raw parameters as well, so you can pass anything that's supported. how those parameters work and their syntax varies for each control. There is some built-in documentation in the Photino.NET methods and properties descriptions as well as issues here containing links to docs for each control.

This may also be solvable using Javascript's internal APIs, see here https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/play#usage_notes.
Closing this due to inactivity


Update to dunfell 23.0.7

@resin-jenkins test this please

We switched to Honister so we can close Dunfell related PRs


Return the civiform-eng config back to pre-northstar settings

The change in this commit should be undone after northstar launches. This is mean to facilitate running the northstar tests alongside tests of non-northstar in order to keep main staging unblocked for weekly releases.
https://github.com/civiform/civiform-staging-deploy/commit/2298f9a3f7bd8043c4867bd2029489715f56c171

Related #7266


Add dtime parameter to read_ptu function

Description
Return number of bins in one period (1/frequency) by default.
Requires ptufile >= 2024.2.20.
Release note
Summarize the changes in the code block below to be included in the
release notes:
Add dtime parameter to read_ptu function

Checklist

[x] The pull request title, summary, and description are concise.
[ ] Related issues are linked in the description.
[x] New dependencies are explained.
[x] The source code and documentation can be distributed under the MIT license.
[x] The source code adheres to code standards.
[x] New classes, functions, and features are thoroughly tested.
[x] New, user-facing classes, functions, and features are documented.
[ ] New features are covered in tutorials.
[x] No files other than source code, documentation, and project settings are added to the repository.

Hi @bruno-pannunzio
This PR uses a feature of the latest ptufile library to return the number of bins in one period by default. Formerly, the number of bins with photons was returned. If you use the ptufile library directly, use dtime=0.

I think it still shows a little bit of the artifact

Yes, that's what I was mentioning a while ago in another meeting. I have seen these artifacts at the end and beginning only in PTU files exported by Leica software. I think they might come from slight discrepancies in the sync of micro and macro clocks. Or maybe they are due to the algorithm used to write photon times to PTU? Anyway, looks like we can't do much about it.


cmd: Trim --core.grpc flag value

Resolves #766.

Codecov Report

Merging #768 (1d51f97) into main (45f23d8) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #768      +/-   ##
==========================================
- Coverage   52.72%   52.72%   -0.01%
==========================================
  Files         116      116
  Lines        6596     6598       +2
==========================================
+ Hits         3478     3479       +1
- Misses       2754     2755       +1
  Partials      364      364

Impacted Files
Coverage Δ

cmd/flags_core.go
47.72% <0.00%> (-2.28%)
:arrow_down:

fraud/bad_encoding.go
31.11% <0.00%> (-3.34%)
:arrow_down:

das/daser.go
76.31% <0.00%> (+2.63%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45f23d8...1d51f97. Read the comment docs.

I've just included your suggested change @Wondertan . This however means that the endpoint MUST be passed with a preceding http:// https:// or tcp:// in order to be parsed properly by the net/url pkg.
This means all docs with --core.grpc flag must be updated from --core.grpc <ip>:<port> to --core.grpc http://<ip>:<port> @YazzyYaz

Hi @renaynay does this apply also to --core.remote?

Hi @renaynay does this apply also to --core.remote?

@YazzyYaz core remote takes the protocol already so yes

Let me push though to ensure parsing there too


Backport #5000 to 0.11.x

Backport for https://github.com/inventree/InvenTree/pull/5000

Wrong target branch


Estimate the size for the zkOracles circuits

We need to estimate a size for the circuit given #12246 and #12245.
The outcome of this task should be an estimate for the number of constraints how many recursions are needed in Pickles to make it work (this is, if we can make it work by splitting the circuit containing the primitives and TLS transcript).

It is now ready to estimate.

Closing this task as we have agreed on the approach to it.


Allow cf uups to merge existing credentials

cf uups only sends newly set credentials and clobbers existing credentials, as described in #557. IMO, this is a counter-intuitive behavior, and has caused some confusion on my team. I think adding the option to merge existing credentials would be useful here. I wrote a cli plugin to do this at https://github.com/jmcarp/cf-mups, but I think it would make sense to incorporate credential merging into this repo instead of using a plugin.

Hi @jmcarp thanks for creating this issue. We've updated the uups help text to be explicit about credential overrides. However, we will not plan on making any further changes to enhance the behaviour of this command.

@stipx I've heard this issue from several people and created an issue to look into it further: https://www.pivotaltracker.com/story/show/156767059

@stipx Just to clarify are you talking about viewing the credentials of a user provided service instance or a service instance backed by a service broker?

@Samze imagine you have a buildpack integration for some service. Because of wrong credentials it is not able to start up so I'm not able to show the environment variables via cf env <appname>.
The integration is using a user defined service for the credentials.
When I just want to update the wrongly entered endpoint URL (typo or whatever) and want leave the password and username unchanged I have to look them up as well just to enter the endpoint again.
So just printing the credentials of a service would help because then I could just print the credentials and copy them over in my update call.
Hope that clarifies the use case.


Devious Dungeon 2 [PCSB01342]

App summary

App name: Devious Dungeon 2
App serial: PCSB01342
App version: 1.00

Vita3K summary

Version: v0.1.8
Build number: 3230
Commit hash: https://github.com/vita3k/vita3k/commit/c571602e
CPU backend: Dynarmic
GPU backend: Vulkan

Test environment summary
Tested by: space1
OS: Windows 11
CPU: AMD Ryzen 5 3600
GPU: Nvidia 3070
RAM: 32 GB
Issues
Freeze on loading.
Screenshots

Log
vita3k_save.log
Recommended labels
Bootable

i highly doubt it's bootable, it's more like nothing

Vita3K summary

Version: v0.2.0
Build number: 3691
Commit hash: https://github.com/vita3k/vita3k/commit/31826697
CPU backend: Dynarmic
GPU backend: Vulkan

Test environment summary

Tested by: Slade
OS: Microsoft Windows 10 Professional (build 19045)
CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
GPU: NVIDIA GeForce GTX 1060 6GB
RAM: 16 GB

Issues

No changes to this game, it still crashes shortly after the splash screen.


Warn and offer to fix iRT database if target is a standard and also in the library

Since there isn't any user interface involved, it might be simpler if you move the checking for duplicates and deleting them to "IrtDbManager.LoadBackground".
That way, all of the database reading/writing is happening on a background thread and you don't need to worry about what might happen if the Document changes in between calls to "SetDocument".

Thanks, those are all good suggestions. I made the changes.


Secret Update changed from data to resource HCO-77868

Thanks Bill.
Once this is merged and released there needs to be respective updates noted in the searches below

https://gitscm.cisco.com/plugins/servlet/search?q=project%3AHCI conjur_secret_update
https://github.com/search?q=org%3AHybridCloudIT+conjur_secret_update&type=code

Tag @brbharad for noting changes at https://gitscm.cisco.com/projects/HCI/repos/terraform-acme-cert/browse

Bump that release version to 0.7.0 too please


Update cockatrice to 2017-04-15-Release-2.3.16,Thopter_Pie_Network_Revision2

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Thank you for the contribution. It was merged directly as 9958c5c1171aae793ebe51e42f4508f5a729bc02. Your contribution is still credited to you.


Paket generates odd binding redirects

Description
When referencing MathNet.Symbolics, paket generates an assembly binding redirect for System.Runtime.Numerics to <IP_ADDRESS>. I can't see any reason for this. MathNet.Symbolics uses <IP_ADDRESS>. In fact, I don't even see that version of System.Runtime.Numerics on NuGet.
Repro steps

Create a new F# class library project targeting netstandard2.0.
Initialize paket, set redirects:on, and set it to reference MathNet.Numerics.
Add an App.config file to the project.
Run paket install.

This contains an example of the problem: test.zip
Expected behavior
I expect paket not to generate assembly binding redirects in this case. If they are needed, I expect the redirect for System.Runtime.Numerics to target <IP_ADDRESS>.
Actual behavior
An assembly binding redirect is generated in app.config for System.Runtime.Numerics to a version that isn't referenced anywhere, and doesn't appear to even exist on NuGet. This may cause errors at later points.
Known workarounds
Disable redirects in paket, I suppose?

The version in the binding redirect is the assembly version, which isn't necessarily the same as the package version. System.Runtime.Numerics <IP_ADDRESS> has a binary for netstandard1.3, which has an assembly version of <IP_ADDRESS>. So the binding redirect is correct.
Anyways, libraries don't need binding redirects. The final application does. See this question on SO: https://stackoverflow.com/questions/48377474/do-binding-redirects-in-app-config-for-class-libraries-do-anything

If the binding is correct, then why does the compiler emit MSB3277 "Found conflicts between different versions of the same dependent assembly that could not be resolved"?
Regarding redirects in libraries, the repro sample is simplified, but in the real case I have a large repository that includes both libraries and executables. They share a paket.dependencies file in the repository root. This file has redirects:on, so redirects are generated for all projects regardless of whether they are libraries or executables. Is there any way to tell paket to only generate redirects for executables?

It's hard to tell without seeing the full picture. Does the warning complain about this particular assembly? Does the project contain an App.config file? Does it contain an entry for this assembly? Which framework does the project target?
Paket should create binding redirects in any App.config file it finds. Maybe try to set redirects to force?

If it helps, the repro which is linked in the original post demonstrates the problem exactly.
To answer your questions:

the MSB3277 warning is indeed complaining about System.Numerics specifically.
the project does contain an App.config file.
the App.config file does contain a paket-generated entry for this assembly.
the project targets netstandard2.0.

I don't think setting redirects to force should make any difference. My problem isn't that redirects are absent; it's that they are wrong. If I delete them, the warning goes away. Just in case, I have tried switching to force, and I observed no difference.

It looks like MSBuild resolves to System.Runtime.Numerics <IP_ADDRESS>, which is included in the .NET Standard, instead of the assembly in the System.Runtime.Numerics package (version <IP_ADDRESS>), for which Paket generates the assembly redirect.

Interesting. If I run msbuild with verbose log, I find this:
Could not locate the assembly "System.Runtime.Numerics, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.
It looks like this causes MSBuild to fall back to <IP_ADDRESS>, with the warning.
Any idea why it wouldn't find the assembly? It's a dependency of MathNet.Symbolics, so paket should have downloaded it somewhere?


genpy: 0.5.9-0 in 'jade/distribution.yaml' [bloom]

Increasing version of package(s) in repository genpy to 0.5.9-0:

upstream repository: git@github.com:ros/genpy.git
release repository: https://github.com/ros-gbp/genpy-release.git
distro file: jade/distribution.yaml
bloom version: 0.5.21
previous version for package: 0.5.8-0

genpy
* warn about using floor division of durations (#58 <https://github.com/ros/genpy/issues/58>)
* allow durations to be divided by other durations (#48 <https://github.com/ros/genpy/issues/48>)
* avoid adding newline in msg_generator (#47 <https://github.com/ros/genpy/issues/47>, #51 <https://github.com/ros/genpy/issues/51>)

Waiting for other releases...


Write a readme on an example build workflow in gulp

References Issue #9

I made the requested changes and polished the readme

I made the requested changes and polished the readme. Let me know if you spot any typos that I may be blind to.


feat: make file is ready

What type of PR is this?
/kind feat

What is the purpose of this PR and its related background?
make file is ready
Which issue(s) this PR fixes:

Fixes #
Does this PR introduce a user-facing change?

NONE

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 85.63%. Comparing base (b5ebe7c) to head (3e05e99).
Report is 6 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #90   +/-   ##
=======================================
  Coverage   85.63%   85.63%
=======================================
  Files          45       45
  Lines        1211     1211
=======================================
  Hits         1037     1037
  Misses        174      174

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Python: Bump Python version to 1.1.0 for a release.

Motivation and Context
Bump Python version to 1.1.0 for a release.

Description
Bump Python version to 1.1.0 for a release.

Contribution Checklist

[X] The code builds clean without any errors or warnings
[X] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
[X] All unit tests pass, and I have added new tests where possible
[X] I didn't break anyone :smile:

Python 3.10 Test Coverage Report • FileStmtsMissCoverMissingTOTAL636371889% report-only-changed-files is enabled. No files were changed during this commit :)
Python 3.10 Unit Test Overview

Tests
Skipped
Failures
Errors
Time

1511
1 :zzz:
0 :x:
0 :fire:
23.442s :stopwatch:


chore: Update supported versions

Add missing changes to officially support v1.30

Updated welcome messages
update envtests

Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes https://github.com/kedacore/keda/issues/5828

/run-e2e internal

/skip-e2e


[Bug] [Standalone] Error uploading binlog job to standalone cluster

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
My standalone enabled the HA configuration，When I uploaded the binlog job for the second time, the task reported an error
The error appears to have occurred in jobmanager

What you expected to happen
The job runs no matter how many times it is submitted
How to reproduce
just launches the standalone cluster of two JobManagers, three TaskManagers, and uploads binlog jobs twice
Anything else
No response
Version
master
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

PR merged. #1339


RAG enhancement. Add chroma, qdrant, etc vector databases collection context to queries

Describe the solution you'd like
I am a little new to AI so I am not sure about this direction or even if this is already implemented and I missed it, I can potentially help with the development too
I didn't see qdrant, chromadb etc options to add into the rag system. if I/someone added in these would the following config options and aichat options work?
config
vector_database:
  - name: my_db
    database_type: chromadb
    connection: localhost:8000
    default_collection: all_collection

cli
aichat --vector-database my_db --vector-database-collection other_collection

AIChat has its own vector database implementation and does not rely on third-party vector databases.

@sigoden
It does, but it's quite limited in terms of how the documents are vectorized. It'd be convenient to fetch, process and insert the data using other frameworks into another vectorDB, to which one can query from the command line using aichat.


Add PHP 8.2 in CI tests

What was changed
Added PHP 8.2 in CI tests

About PHP 8.2 and Windows:
the gRPC extension can't be installed on this setup yet

It can be related with unavailability of gRPC binaries for windows.
I've created an issue about it ( https://github.com/grpc/grpc/issues/32304 )

Related: https://github.com/protocolbuffers/protobuf/issues/11452#issuecomment-1420236110


Added Contributing.md

Impressive. Thanks for contributing!

Next time when you're making a pull request connected to an open issue, add a line that says "Closes #3" where 3 is the issue number.
That way the issue is automatically closed after the pull request is merged.

Alright Noted!

kindly edit the hacktoberfest tag as it has a typo.

Thanks for pointing it out.