[Feature Request] Test out SWB upgrade in Dev/Sandbox env

We will want to test out the upgrade SWB version (with TRE and R-server workspace) under Dev/Sandbox environment to prep for the upgrade in production

I'll be able to begin working on deployment of this under Sandbox, this coming Monday :)

Pushing this to January, due to other commitments this week with UMKC ISFS projects

This is a duplicate of #10 so closing it and copying updates to there

@fergusonsha-umkc this is not a duplicate. #10 is the root ticket laying out the entire plan of performing the upgrade (assigned to me to oversee), this is one of the specific task of just testing the upgrade out in Sandbox.

@fergusonsha-umkc  any updates on this?

Not yet.

@fergusonsha-umkc any updates on this?

As soon as I am able to focus on this, and it will need dedicated focus not multitasking, I will.  There are start-of-semester obligations I have to fulfill for UMKC, before I can switch mental gears back here.  I'll provide updates, once I have something, tmeanwhile please rust that it's not been forgotten as a priority.

He has no prior experience with deploying Service Workbench from its Git codebase, however he will be involved.  Our plan has been to have me do Sandbox and work through any wrinkles, then I will guide him with doing it hands-on himself in Dev, to get the experience.

I aim to start working on Sandbox upgrade and switch to TRE, this coming Monday.

Deploying Cloud9 environment for Sandbox SWB upgrade

Codebase pulled to latest release, looking for the most recent upgrade documentation to avoid assumptions.

Using https://github.com/awslabs/service-workbench-on-aws/blob/mainline/docs/Service_Workbench_Installation_Guide.pdf

C9 install source updated to latest 4.x release, verified config files are still set, running the environment deploy script per upgrade instructions ...

----- ENVIRONMENT DEPLOYED SUCCESSFULLY 🎉 -----
Doing post-upgrade CFN steps

The new ALB RStudio Server instructions are rather more involved than just rebuilding a new AMI, looks like.  Dedicated section for it.

40 changes including addition of AppStream components, being made to the 'umbmiaws-swb-onboard-account' stack as part of the upgrade

Hmm, stack update failed... troubleshooting

Reason for failure:

2022-01-31 14:57:33 UTC-0600
AppStreamStack
CREATE_FAILED
null (Service: AmazonAppStream; Status Code: 400; Error Code: AccessDeniedException; Request ID: 14ec11eb-14fd-4c44-8837-45ca35ff7673; Proxy: null)

Going to AppStream and seeing if it has to be enabled first, maybe

Oh, crud...
Region Unsupported
AppStream 2.0 is not available in US East (Ohio). Please select another region.
So.... hmm.  I will ping Travis and Nathan about when AppStream will be supported in our region, and ask for ideas on how we can disable it in the meantime to at least have the rest of the upgrade to play with...

Email sent:
Hi Travis, Nathan,
I was working on upgrading SWB in our Sandbox environment today, and everything was going pretty smoothly until I tried the onboard account stack update, which failed out when attempting the AppStream stages.  When I went to AppStream directly in our AWS console, to see if maybe it needed to be activated/enabled for the account, I was presented with this rather dismaying message:
Region Unsupported
AppStream 2.0 is not available in US East (Ohio). Please select another region.
Is there any way to get an ETA on when AppStream will become available in our region so we can use the Trusted Research Environment feature, and in the meantime, can we toggle it off in order to take advantage of the rest of the upgrade's features and bug-fixes?  If we need to hop on a Zoom session to discuss, we can arrange that.
Thank you!

Travis responded:
Let me get with the service team and ask about AppStream in us-east-2.
As far as TRE goes, that feature is a one-way door once it’s fully deployed.  (In other words, once you turn it on, you cannot turn it off.)  If it wasn’t completed, you can try setting things to false in the configuration document and re-deploy.

I'll attempt his suggestion about turning it off, tomorrow.

Re-running the stack update with AppStream disabled

Stack update was successful with AppStream disabled, working on getting logged in to test it out.  Login page, for Sandbox at least, has changed a bit.

OK, got logged in as<EMAIL_ADDRESS>using Cognito Native Pool, and it does look like we have AWSSO accounts set up, so I will need to debug why those aren't working now...

Also need to do the AMI rebuilds...

I'm going to back-port our configurations and customizations from Dev to Sandbox, which should fix some things like SSO...

OK, had to reassign the IdP::UserPool attributes to get SSO working again.

Also having to repopulate the VpcId and SubnetId in the machine-images/config/infra/*.json

Removed old stock AMIs/Snapshots and building upgraded machine images for Linux, Windows, EMR ... RStudio is handled separately

Update from Travis:
"As for AppStream, it looks like it is in private beta for us-east-2.  The team didn’t have a target date yet, but they do have some customers who are using it.  If this is something you would like to pursue, we would need to get an NDA in place with you and then we could invite you into the private beta, as well.  So, think about that one for a while and let me know what you would like to do, if anything."

To expedite things, I will just set up one configuration for each workspace type, which we can adjust instance type for, as desired.

Nice!  Given this, I might hold up a bit on importing workspaces until we get AppStream turned on :)

Nice, the template variables are pre-populated now :)

Travis update:
"Let me get back to the service team and ask what the process is for the private beta.  It’s usually just a matter of knowing what account to add to an allow-list.  I didn’t realize you had the NDA in place already.  (Thanks, Matt!)  I will get back with you soon."

Sagemaker workspace tested successfully

Error provisioning environment shaun-test-upgraded-Linux. Reason: AmazonCloudFormationException Parameters: [EgressStoreIamPolicyDocument, SolutionNamespace, IsAppStreamEnabled] do not exist in the template (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: 4c0459f7-72c9-49df-b6af-56c52f7d622f; Proxy: null)
So I will wait until we have AppStream available to proceed with the rest of this.

Travis update:  "Your accounts have been added to the allow-list.  You should be good to use AppStream in the us-east-2 region in those accounts.  Deploy away!"
I will resume this soon as I can.

@fergusonsha-umkc Let's request a demo presentation from Travis regarding AppStream

Current status of implementing TRE in Sandbox:
Hey Travis, I eventually found the AppStream documentation (its not part of the Installation Guide PDF yet, looks like) under docs/docs/post_deployment/appstream.md and its helped get me further along on enabling this, but not by much unfortunately, I'm already running into a problem again... we might need to set up a working session in Zoom sometime in the coming week to go through this in realtime?
When running the commands to kick off the AppStream Image Builder, I ran into this problem (maybe because we don't have a default VPC defined in our accounts, which I had to define in the AMI image .json files for those to build, but its not as clear to me how to do that in this .js file)...
AWSReservedSSO_AWSAdministratorAccess_8d0d41bd23adb7bf:~/environment/service-workbench-on-aws/scripts/app-stream (mainline) $ npm run start-image-builder -- default us-east-2 default default

<EMAIL_ADDRESS>start-image-builder
node -e "require('./start-image-builder.js').runCodeAsScript()" "default" "us-east-2" "default" "default"

Starting Image Builder using AWS Profile default in region us-east-2 with base image AppStream-WinServer2019-10-08-2021 and instance type stream.standard.medium
Starting image builder with name SWBImageBuilder-1645723647265
Failed to start image builder TypeError: Cannot read properties of undefined (reading 'VpcId')
at StartImageBuilder.createImageBuilder (/home/ec2-user/environment/service-workbench-on-aws/scripts/app-stream/start-image-builder.js:63:42)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async StartImageBuilder.run (/home/ec2-user/environment/service-workbench-on-aws/scripts/app-stream/start-image-builder.js:37:7)
I will continue to troubleshoot it while waiting for a response, too.

Hard-coding the VpcId worked to get image builder kicked off:

npm run start-image-builder -- default us-east-2 default default
<EMAIL_ADDRESS>start-image-builder
node -e "require('./start-image-builder.js').runCodeAsScript()" "default" "us-east-2" "default" "default"
Starting Image Builder using AWS Profile default in region us-east-2 with base image AppStream-> WinServer2019-10-08-2021 and instance type stream.standard.medium
Starting image builder with name SWBImageBuilder-1646068218369
Waiting for Image Builder to finish starting up. This can take 5 to 10 minutes
Image Builder is in PENDING state

Status update, after encountering lots of contradictory and just plain not-working scenarios with the Sandbox upgrade yesterday:  I've backed the git source tree down to v4.3.1 which is latest release of version 4, and did more of a clean-slate install to ensure all the components are in sync with this version release, and today things are looking better... although I do hope to be able to do an upgrade path in the App-Dev environment, following their directions did not work as described/expected for Sandbox :(  I am documenting anything that I've had to do that varies from their documentation, particularly regarding VPCs.  I believe our templates and AMIs will remain functional, though we may have to add an AppStream component to them.  I'll have more information after I get a successful workspace launch under AppStream in Sandbox...

Got a Linux workspace launched under Sandbox SWB v4.3.1 and connected through AppStream, aka the Trusted Research Environment.  I will admit, for as much as we fought to get AppStream, I am wholly underwhelmed by the user experience.  You'll see what I mean.  I'm doing iterative testing of all workspace types, and then will open Sandbox for general testing very soon.

Got a Linux workspace launched under Sandbox SWB v4.3.1 and connected through AppStream, aka the Trusted Research Environment. I will admit, for as much as we fought to get AppStream, I am wholly underwhelmed by the user experience. You'll see what I mean. I'm doing iterative testing of all workspace types, and then will open Sandbox for general testing very soon.

That will be some good feedback to the Travis and AWS SWB team. Also, if the user experience is compromised too much, I think we can probably wait until an improved release is ready, or just deploy it partially for more sensitive data.

OK the Windows workspace connection experience is MUCH better than the Linux one was.

I'll open this for testing once I'm sure all the workspace types launch without issue, and know how to go through the connection hoops for each of them myself, but I'm not calling this "done" done until I get the RStudio piece set up, which is a saga in and of itself in the installation guide now.

All workspaces marked Approved are working in the Trusted Research Environment under Sandbox SWB now.  I'm opening access to Aaron as admin, and Song, Vasanthi as researchers.  This will let you get a feel for the new way workspaces are connected to via AppStream instances, while I continue work on installing the new RStudio backend and instance.  EMR is proving difficult to get launched also, because none of the instance types I set up are acceptable to it, despite being in the valid list for some reason.  I figured RStudio is more important to focus on though.
Please remember this is a fresh-out-of-the-box instance of SWB for Sandbox to get experience with what it entails, our own customizations will be applied when we upgrade App-Dev after all baseline components are vetted in Sandbox.

I'm also looking into how to simplify the login jump page for Service Workbench so AWSSSO is the only (or default) option, although you can use SSO either path you pick, the Cognito path just has the potential for confusion I think.  They integrated Cognito into the Service Workbench deployment script, and are deprecating the DynamoDB-based user logins in favor of Cognito for those who don't have SSO set up.  Our landing page has customization in App-Dev/App-Prod for the usage agreement also, so not a big deal.

Progress report:  got through my first pass at the RStudio-Server instructions, resulting in an AMI and stack which I am currently testing in SWB under Sandbox

Well, the first ACM certificate I set up couldn't be validated, I think because I had an extra character in the corresponding CNAME value.  Generated a new one, double-checked the CNAME value matches without that extra character, and just waiting for it to come out of Pending status, so the workspace stack will launch past that point (it failed due to the certificate not being valid)

We met with Nathan this morning but still couldn't determine why Certificate Manager isn't DNS validating the certificate we need, so upon his advice I am going to enter a ticket through DLT support for Amazon's ACM team to look into it with us.  All our own testing seems to show that the ACM DNS validation service should be able to get a CNAME response from our hosted zone records for its validation check, but, there also may have been a missing step in the RStudio ALB deployment documentation (a lot of fundamental assumptions get made in their documentation that don't all apply in our Control Tower governed organization structure situation).  Meanwhile, all of the rest of the SWB 4.x upgrade is ready to be tested out, which Vasanthi said she would do perhaps with some student workers.  My recommendation would be for us to proceed with the upgrade in our Dev environment, to prepare for a production upgrade and test with our custom assets, and get the RStudio component in place as soon as this one issue is solved.
Also, Nathan mentioned, he and/or Travis will be reaching out to us soon regarding the next version of SWB which apparently has an even bigger paradigm shift involved in how it would operate... I am trying to get more details.

DLT got in touch with me, and I've reached out via email for approval to upgrade support tiers or seek alternative avenues:
Shaun, thanks again for taking my call.
As we discussed, upon review of account<PHONE_NUMBER>66, this account currently has a Basic Support Plan associated with it, which unfortunately doesn't have access to the Technical Support option needed to pursue engagement with the AWS Technical Support team.
Furthermore, as an Internet2 organization you have control over your support levels at any given time, if you would like to continue this ticket with the AWS Technical Support backline team you may upgrade your support plan to the Business Level Plan (which you can always change back to Basic if you choose to do so)
You mentioned that you will need to discuss this with your colleague's, so we will be on standby for your  feedback on how to proceed
Respectfully,
Michael Davis
DLT Confirmed Stateside Support

Nathan got some feedback from the ACM team that proved key to understanding what's going on.  We had not registered the domain under which the certificates were being created, yet, due to some Service Workbench deployment instruction gaps, but we have the nextgenbmi.umsystem.edu domain that we will use.  Based on the documentation, it seemed like this would be an internal-only DNS domain managed by Route53's Hosted Zone, but it needs to be an actual registered Internet domain (in hindsight, obvious).  So I will proceed with working with DoIT to get the necessary pieces registered with them before creating the hosted zone in Sandbox.  I've deleted the prior-created hosted zone and certificate until then.

Service Request 1079808 has been created for CNAME creation to validate the ACM certificate using the nextgenbmi.umsystem.edu domain

Quick response!

I will proceed this week on testing and tuning the RStudio workspace, and next week I believe Aaron and I can begin doing the upgrade in our App-Dev environment to introduce our customizations... I might, however, introduce our customizations into Sandbox first and get things all worked out there, so we still have App-Dev available for any "needs testing before production implementation" in our current environment.  Then when we are able to have a maintenance window of 2-3 days, we can upgrade our dev and prod instances.

Apparently the entire SWB instance needs to have a custom domain name in order for the RStudio Server workspace to launch, so I'm getting sandbox.nextgenbmi.umsystem.edu registered and certified, and will redeploy using that, and cross all my fingers.

The new custom domainname of sandbox.nextgenbmi.umsystem.edu is now set up and working, hopefully this will do the trick for RStudio Server to be happy.  I fixed the Cognito and SSO application integrations to use this, and now they are happy.

Testing Report: 04/27/2022

Onboarding message and do we need to keep the "Cognito native pool" option?

Sagemaker workspace via AppStream 2.0

longer waiting time at every launch (not just the initial time)

freezes a lot (try pip list)
can't do pip install

@sxinger It would be great if we can track the testing in a separate ticket. I created this ticket for you https://github.com/Missouri-BMI/service-workbench-on-aws/issues/22

Next steps: @sxinger will reach out to Travis and setup accounts for Travis/Natan in our Sandbox environment.

We have decided the next steps with Travis and Nathan. Follow up tickets have created.


Developing a measurement counter using batch queries

Hello, I am a new user to Kapacitor, and I am trying to develop a counter for a measurement based off of the measurement incrementing over time. To do this, I decided to use batch queries. I made two queries for the same measurement with a time offset applied to one of the queries, which in theory would allow for a comparison statement to be made between the two queries, as a ‘!=’ statement would trigger an alert if the two query values are different, due to the time offset. I’m sure I am just missing some minor syntax queues, as I am new to the service, but the script currently sends out alerts every 30s rather than every time the measurement increments. Does anyone have any ideas on how to implement this counter functionality?
// Template Author: Benji Smith
// // Kapacitor Notifications for doing alerts for counting mnemonics Template ////
// This script has been developed to work with only one counter mnemonic in order to get the SCENG team comfortable with using the TICKscript syntax

// Name the script, this will help identify errors in log messages if errors occur
var name = 'Name'

var data = batch
    // query the database for the mnemonic in a batch format, to keep the code latency to a minimum.
    |query('SELECT "raw" AS "value1" FROM "My-Database"."inf".Measurement')

        .period(1m)
        .every(1m)
        .offset(-3m)
    |last('value1')
        .as('value1')

var pastdata = batch
    // query the database for the mnemonic in a batch format, to keep the code latency to a minimum.
    |query('SELECT "raw" AS "value2" FROM "My-Database"."inf".Measurement')
        .period(1m)
        .every(1m)
    |last('value2')
        .as('value2')

// This join command combines the query to make the two queries cross compatable
var joined_data = data
    |join(pastdata)
        .as('data', 'pastdata')
        .fill(0.0)
        .tolerance(15s)

// This alert command does the counter comparison and sends message to designated slack user/channel
// performance_error
joined_data
    |alert()
        .crit(lambda: "data.value1" != "pastdata.value2")
        .message('The measurement has incrimented, value is now {{ index .Fields "data.value1" }} from {{ index .Fields "pastdata.value2" }} at {{ .Time }}  ')
        .slack()
        .workspace('Direct To Benji')

The answer to this question is at:
https://community.influxdata.com/t/developing-a-measurement-counter-using-batch-queries/18657/6


timer: fix timer leak in case of odp_timer_set_abs() failure

Signed-off-by: Matias Elo<EMAIL_ADDRESS>

Reviewed-by: Bogdan Pricope<EMAIL_ADDRESS>Best regards,
Bogdan


Site down: atelierdunoir

Seems gone to mangadex and googledrive (See https://atelierdunoir.wordpress.com/)

Description
Data

Url down
http://www.atelierdunoir.info/

Thank. I will remove the site from the supported.
As for the new location. Sorry, but I have no plans to add support for google drive
This is excessive complexity.
Perhaps one day I will return to this question, but not now

Thanks. I will remove the site from the supported.
As for the new location. Sorry, but I have no plans to add support for google drive
This is excessive complexity.
Perhaps one day I will return to this question, but not now

Removed


Graph View: User can view/manage relationship between vClusters and physical clusters

UX Design:
https://www.figma.com/file/we6QSscpQH6MkXilCBO9aT/Kubefirst-Console?type=design&node-id=6667-72463&mode=design&t=5f1EXs1sSXqmLtEx-4


Run validation before any commands start running

Let's say I have the following commands:
file-read test.json | transform | file-write test.csv

In this use-case, let's say I want to validate the file extension before running a command. I add the validate method, but it won't run until after the whole file has been read in and transformed. In my use-case, that may take hours.
I'd like the validation to run immediately before any command starts running.
@dthree Any thoughts?

It should run for a single command. Are you saying that the validate for specifically the piped commands doesn't happen? If so, yeah, that's a bit tricky. You would have to throw it in right around here:
https://github.com/dthree/vorpal/blob/master/lib/vorpal.js#L919

Thanks for the pointer. I am going to implement this weekend. I need it.
My tool, each command can take several hours - days even. It renders a progress bar... but I really need to validate all commands in advance. Some of the validation involves HTTP calls.

K got it.


AWS::MSK::Cluster.ClientAuthentication.Sasl

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-msk-cluster-clientauthentication.html

Thanks!


Expose is_minimized and is_visible from Winit

What problem does this solve or what need does it fill?
I think right now the only way to access those Winit APIs from Bevy is to get NonSendMut<WinitWindows>.
This is the only way to know if a window is minimized/maximized, since there is not even an event for them so it's not possible to just keep track of that,  as far as I know.
What solution would you like?
One or several of those:

Have a is_minimized, is_maximized, and is_visible methods on Window, to reflect Winit's method. (note: those are not available for all platforms)
Have a public minimized and visible field on Window, similar to focused, for minimized it could also be mutated to request minimizing/maximizing, replacing the set_maximized/set_minimized methods. (note on this: mutating this field if the request fails might lead to confusing behavior, same for visible that is not supposed to be mutated at all, right now we have similar problems with focused actually since a Window can't be put out of focus, but the field can be set to true with no consequence)
Have events to indicate when a Window is maximized/minimized/visible, if Winit have those (not sure), similar to WindowFocused

What alternative(s) have you considered?
Use NonSendMut<WinitWindows>, and potentially document it for set_maximized/set_minimized methods.

I exposed the visible property in this PR https://github.com/bevyengine/bevy/pull/9355

Window State & Visiblity
New to Bevy so I am not sure if these struct/enums already exist but I would imagine the window state and visibility status could be represented with something like:
enum WindowState {
    Normal,
    Maximized,
    Minimized,
    Fullscreen,
}

enum WindowVisibility {
   Visible,
   Hidden,
}

I am a fan of using enums over booleans, even in the case of WindowVisiblity which allows future expansion of other states without needing to refactor.
For the case of WindowState, there are some platform specific things that should be taken into account. On Windows, you have the Maximized behavior, whereas on macOS its Fullscreen instead. Would this be captured as "maximized" or "fullscreen"?
Or would the "fullscreen" behavior be more a separate enum for the traditional video game behavior of "windowed / borderless fullscreen / exclusive fullscreen"?
Window Events
I second the idea of exposing various window events for when these things change. Assuming the above, something like WindowStateChanged and WindowVisiblityChanged which exposes the new enum value.
You could have convenience events which are also emitted for WindowMinimized, WindowMaximized and WindowRestored (from min/max -> normal), for when you don't care about checking the enum value itself.
Request User Attention
Would also be good to expose the request_user_attention functionality.
In Windows this flashes the icon in the taskbar, in macOS the dock icon bounces until the user clicks it. I assume Linux desktop environments have a similar flash or attention getting mechanism.
This could be done with an event writer perhaps EventWriter<AppUserAttention>.

I'll try to open a PR that exposes the minimized and maximized statuses of the windows as boolean (just to mimic what the winit API returns). I'll keep it simple and just forward the values exposed by winit. I'll take a look at the events and the user attention feature later.

You can take a look at https://github.com/bevyengine/bevy/pull/9815. I made it as concise as possible. I still have to test it properly, but I'd like some early feedback on the code as I'm not very convinced on the implementation.

I'm considering blocking or closing my PR until winit adds events for minimize and maximize actions (check the following issue). I was able to programatically expose and change these statuses in my machine (e.g. set a maximized flag in the primary Window and update the underlying winit window), but the problem is that I don't have (or know) a way to react to actual user input due to the lack of these events in the windowing backend.
A potential solution that I can think of would be to have an independent system check the winit status periodically and update fields in the Window component, but it feels suboptimal and might trigger other systems that check for Changed<Window> unnecessarily.
I've also checked the WinitWindows resource but I'm not sure if it's possible to check it for changes (and if window status changes will actually trigger them) since it's a NonSend resource. Also feels overcomplicated even if possible.
Any input on this will be welcome, but I think the implementation will be incomplete under these circumstances.


Adding some Twitter screenshot for TanyaAja share

Is your feature request related to a problem? Please describe.
Adding new section to show how user sharing some questions in Twitter.
For marketing purposes, add it in the homepage
Describe the solution you'd like
Find way how to embed Twitter without sacrificing performance too much
Maybe generated in the build-time, since it's not needed to be interactive and showing real stats.

https://vercel.com/blog/introducing-react-tweet

https://twitter.com/lynxluna/status/1712849661550329929
https://twitter.com/pveyes/status/1713003572634485039
https://twitter.com/Maz_Ipan/status/1712776713963450491
https://twitter.com/th_clarence/status/1712441248190185880
https://twitter.com/mgilangjanuar/status/1711254574244520075


[FLINK-18134][FLINK-18135][docs] Add documentation for Debezium and Canal formats

What is the purpose of the change
Add documentation for Debezium and Canal formats and also update HBase and JDBC docs to have better DML examples.

Thanks for the update, LGTM now.


Doctor's AppPath might be different when run inside Docker

I run gitea with this command:
docker run -d --restart=always --publish 3000:3000 \
--publish 22:22 --network gitea \
-v /opt/gitea:/data \
-v /etc/timezone:/etc/timezone:ro \
-v /etc/localtime:/etc/localtime:ro \
-e USER_UID=1218 -e USER_GID=1218 \
-e GITEA_WORK_DIR=/data/gitea -e GITEA_CUSTOM=/data/gitea/custom \
--name=gitea gitea/gitea:latest

The AppPath is /app/gitea/gitea and such is written to authorized_keys, however when executing docker exec -ti gitea su git bash -c "cd /data/gitea/log && gitea doctor" the AppPath is /usr/local/bin/gitea. This issue leads to doctor always complaining that authorized_keys is out of date.

This can be mitigated by using docker exec -ti gitea su git bash -c "cd /data/gitea/log && /app/gitea/gitea doctor", i.e explicitly providing /app/gitea/gitea as path

This can be mitigated by using docker exec -ti gitea su git bash -c "cd /data/gitea/log && /app/gitea/gitea doctor", i.e explicitly providing /app/gitea/gitea as path

I suspect it would be easier to change the docker PATH to place /app/gitea/ on the PATH before /use/local/bin
Another option would be to change the docker exec to run /use/local/bin/gitea instead of /app/gitea/gitea but I think that would no doubt cause other issues with data paths etc.


Add support for logging if test fails

From<EMAIL_ADDRESS>on December 17, 2012 12:35:31
Now you can do e.g. log().ifError() but that only looks at status code. So for example if you expect 400 and log().ifError() and receive a 200 then nothing will be printed.
Original issue: http://code.google.com/p/rest-assured/issues/detail?id=212

From<EMAIL_ADDRESS>on January 23, 2014 07:40:21
To me it looks like it would suffice to add something along the lines of
logWith(new ResponseLoggingFilter(getPrintStream(), isNot(responseSpecification.getStatusCode())))
to ResponseLogSpecificationImpl. Unfortunately I seem to be unable to build rest-assured here at the moment to verify that.
Thoughts?

From<EMAIL_ADDRESS>on January 23, 2014 10:15:25
That's a really nice solution! Too bad I just released a new version otherwise I could have included this. Thanks for the tip!

From<EMAIL_ADDRESS>on January 31, 2014 16:10:32
Would love this to be supported in all given implementations

From<EMAIL_ADDRESS>on January 31, 2014 17:04:14
To clarify, I run into this problem when I try to use given( RequestSpecification, ResponseSpecification )
I use the ResponseSpecBuilder to build out my ResponseSpecification with ResponseSpecification = ResponseSpecBuilder.build().  The problem seems to be that ResponseSpecBuilder doesn't support log(LogDetail.ALL) like RequestSpecBuilder does.
At any rate... to work around this and log the service response I do this beautiful hackish typecast:
String json =
((TestSpecificationImpl)given( reqSpecification, resSpecification )).getResponseSpecification().
log().all()   or     log().ifError()    or the proposed logWith()   .
get("your_api").asString();
You still get the AssertionErrors as before, but you at least get the response body logged out.  If you really want to hack around the long stack trace for clean test output (which I don't recommend):
try {
RestAssured given() . get() / put() / post() / head() / etc
} catch ( Throwable t ) {
print nicely
use a java or junit assert?  throw t?
}

From<EMAIL_ADDRESS>on February 09, 2014 07:10:51
@jmagnus: That's a separate issue. Please report it as a new issue that we should add logging support for ResponseSpecBuilder. I remember that I tried to do that once but for some forgotten reason I never pulled through.

From<EMAIL_ADDRESS>on March 25, 2014 07:35:19
@ wolfgang: Unfortunately I don't think your proposal will work since there can be other errors that cause test failure (for example a json path may not be as expected).

From<EMAIL_ADDRESS>on March 25, 2014 07:52:01
Hm, true, my proposal does not cover all possible test failures.
I still think it would be valuable to have something along the lines of log().ifUnexpectedStatus()
It may not be true to the letter of the subject of this bug, but at least in the spirit of the original report, which specifically mentions printing on an unexpected 200.
Although, if you would prefer to handle the general case in this bug and my request separately, I'll gladly file another report.

From<EMAIL_ADDRESS>on March 25, 2014 07:56:59
I'm thinking that perhaps this should be enabled by default? When a test fails both the request AND response is logged to the console (unless configured otherwise). What do you think about this?

From<EMAIL_ADDRESS>on March 25, 2014 08:15:36
In my experience, most of the time Hamcrest's descriptions are fine. Only in the minority of testcases I've writen did I want to see the complete exchange on error.
Although you're right, if that level of detail is needed, I usually want to see both request and response.
Of course, since the general case (logging on any failure) subsumes logging on unexpected return code, I'm fine with either being implemented.
Seeing as you want to make HTTP exchange logging configurable, I for one am fine with your proposal. Maybe one of the other three watchers of this bug want to chime in?

From<EMAIL_ADDRESS>on March 26, 2014 00:46:29
Perhaps it should be turned off by default anyway and we'll wait and see what people think.
I'm still a bit uncertain of how to best implement this feature from an API perspective. Currently you do something like:
given().log().all() to do request logging (by using filters) and ..then().log().all() for response logging. It seem rather strange to me to have a given().log().ifTestFails() that both enables request and response logging at the same time. It's not consistent with the way logging works right now (either with request or response logging). Perhaps it's better to keep it like it is today and allow you do explicitly specify "ifTestFails()" for both the request and response. There can be another way to enable it for both at the same time by using the LogConfig or something:
given().config(newConfig().logConfig(logConfig().logEverythingIfTestFails())). ..
We could also enable a static shortcut for this:
RestAssured.logEverythingIfTestFails();
and maybe:
given().logEverythingIfTestFails() (although this is perhaps a little bit inconsistent?)
What do you think?

From<EMAIL_ADDRESS>on March 26, 2014 05:59:05
I agree. In the interest of consistency and minimizing surprise for the user, I would also prefer to have to specify "ifTestFails()" for both request and response, just as it is with "all()" currently.
LogConfig is, as far as I can see, an appropriate place for such a global switch. I personally don't have strong feelings about the shortcuts one way or another.

From<EMAIL_ADDRESS>on March 26, 2014 06:14:21
Thanks for your comments. I'll see what I can do when I find some time :)

From<EMAIL_ADDRESS>on March 27, 2014 07:44:13
Status: Started

From<EMAIL_ADDRESS>on March 28, 2014 03:54:43
I've implemented support for this now. A new snapshot has been released, depend on version 2.3.1-SNAPSHOT after having added the following repo:
<repositories>
<repository>
<id>sonatype
<url> https://oss.sonatype.org/content/repositories/snapshots/ <snapshots />
</repository>
</repositories>
You can now do like this:
given().log().ifValidationFails() ..
for request logging and:
.. then().log().ifValidationFails(). ..
for response logging.
To enable both at the same time you can use the LogConfig or the following shortcut:
RestAssured.enableLoggingOfRequestAndResponseIfValidationFails();
This has also been implemented for RestAssuredMockMvc.
Please try it out and tell me what you think, I'd like to make a new release soon.
Status: Fixed

From<EMAIL_ADDRESS>on March 31, 2014 05:17:20
Just tried it, looks good to me. Thanks for adding this feature!

Hi @johanhaleby  , how to fail test case forcefully (like Assert.fail()) in restAssured to print fail test case entry in Extent Report.

@sachinguptait Hmm can't you just use Assert.fail() before or after the REST Assured statement?

Thanks @johanhaleby , yes it's working fine, I was doing some mistake.


Added spanish translation

Spanish translation for SLS.
Thanks to @inverse and @luisbandalap!!

Cool! Cheers.


Pattern matching on a lambda?

Awesome project, got to it via HN
I thought I'd try to port microKanren to HVM (why not), but it requires matching on a lambda, or a slight change in the semantics
Here's what I came up with until now, it's likely I may have missed a few things or got them wrong
(Car (Cons x y)) = x
(Cdr (Cons x y)) = y
(Vareq (Var v1) (Var v2)) = (== v1 v2)

(ExtS x v s) = (Cons (Cons x v) s)

(EgInner sc NIL) = MZERO
(EgInner sc s) = (Unit (Cons s (Cdr sc)))

(Eg u v) =
    λsc(EgInner sc (Unify u v (Car sc)))

(Unit sc) = (Cons sc MZERO)
MZERO = NIL

(UnifyInner u v NIL) = NIL
(UnifyInner (Var v) (Var v) s) = s
(UnifyInner (Var u) v s) = (ExtS (Var u) v s)
(UnifyInner u (Var v) s) = (ExtS (Var v) u s)
(UnifyInner (Cons hu tu) (Cons hv tv) s) =
    (let s = (Unify hu hv s);
     (Unify tu tv s))
(UnifyInner u u s) = s

(Unify u v s) =
    (UnifyInner (Walk u s) (Walk v s) s)

(Walk u NIL) = u
(Walk (Var u) (Cons (Cons (Var u) v) s)) = (Walk v s)
(Walk (Var u) .) = (Var u)

(CallFresh f) = @sc(let c = (Cdr sc);
                    ((f (Var c)) (Cons (Car sc) (+ c 1))))
(Disj g h) = @sc(Mplus (g sc) (h sc))
(Conj g h) = @sc(Bind (g sc) h)

(Mplus NIL y) = y
(Mplus (Cons h t) y) = (Cons h (Mplus t) y)
(Mplus fn x) = @(Mplus x (fn))

(Bind NIL g) = MZERO
(Bind (Cons h t) g) = (MPlus (g h) (Bind t g))
(Bind fn g) = @(Bind (fn) g)

EMPTY_STATE = (Cons NIL 0)

// uK ends here

(Zzz g) = @sc(@(g sc))

(ConjL (Cons g NIL)) = (Zzz g)
(ConjL (Cons g gs)) = (Conj (Zzz g) (ConjL gs))

(DisjL (Cons g NIL)) = (Zzz g)
(DisjL (Cons g gs)) = (Disj (Zzz g) (DisjL gs))

(Conde (Cons l alts)) = (DisjL (Cons (ConjL l) (Conde alts)))

(Fresh NIL goals) = (ConjL goals)
(Fresh (Cons v lvars) goals) = @v(Fresh lvars goals)

(Pull NIL) = NIL
(Pull (Cons x y)) = (Cons x y)
(Pull f) = (Pull (f))

(Take 0 f) = NIL
(Take n NIL) = NIL
(Take n (Cons x y)) = (Cons x (Take (- n 1) y))
(Take n f)  = (Take n (Pull f))

//

(Fives x) = (Disj (Eg x 5) (Fives x))

(Main) = (let f = (CallFresh Fives);
          (f EMPTY_STATE))

I also managed to get expansion to diverge in several cases

To pattern-match inside a λ, you need to either create a temporary definition, or create a pattern-matcher for your data. For example,

Oh, I'm so sorry about that. What would pattern-matching on a lambda do, exactly? If I'm guessing right, we could actually kind of have that, now that I think about it. The in-place rewrite rules are compatible, we'd just need to update the parser to allow for it, I guess? But that is so unusual I didn't even consider.

In my case, I would like to match on something being a lambda, and if it is, invoke it

Can you give me an example?

Sure, the Bind pattern can take a thunk, it returns a thunk which calls bind with the thunk invoked:
(Bind NIL g) = MZERO
(Bind (Cons h t) g) = (MPlus (g h) (Bind t g))
(Bind @fn g) = @(Bind (fn) g)

Does that mean that (Bind f g) reduces to (f (Bind (fn) g)) when f is a λ?


Fix editbox focus.

Fixes: #18130
Testing is done locally with multiple profiles working at the same time and sending events like topic edit, stream change, and message edit.

@timabbott would you mind reviewing? Do I send the currently_focused_message_id to a separate commit?

I'm finding this implementation hard to read; perhaps you can take a pass of trying to make it easier to understand, either through comments or refactoring?

Sure, am adding comments along with a commit description for easier evaluation.

@timabbott  updated with a commit message.

@timabbott this has had additional comments since you last reviewed it; ready for another look.

Only rebased it, working to simplify this further. The first commit can be independently merged, though.

@timabbott I pushed a new approach while should work the same as the previous approach.


Implement latest stac 1.0.0 beta

~Note: The tests will fail. This works but is still a draft as it's blocked on other PRs to function (see bottom). But I'm raising it now for visibility as it's been sitting in a branch for a while.~ Done!

Bump stac validation tests from stac 0.9.0 to 1.0.0.beta2 (the latest released beta at time of writing). ( #181)
Support for eo3 documents in stac conversion (#133)
Replace most custom stac-item conversion code with eodatasets3 eo3-to-stac library calls (thanks @sachitrajbhandari!)

Unifying effort means we can hopefully keep it more updated.

Allow query params in /items pages, as with newer stac-api (#150)
Include links to the original odc yaml file in the stac links section.
Includes conversion of legacy eo documents to stac items.
Move all stac URLs consistently under the base of /stac, so they follow stac-api conventions. Adds redirects on old URLs to maintain backwards compatibility.
Support for the ids and collections parameters when querying stac
An optional _full=true query parameter on Stac searches to return full Stac Item metadata within search results. This can be much slower so is not the default. Can be made the default with config property STAC_DEFAULT_FULL_ITEM_INFORMATION = True

Blockages

[x] eo3-to-stac API and improved validation: https://github.com/GeoscienceAustralia/eo-datasets/pull/98

Future ideals

[ ] the mirroring changes in stac-to-eo3 odc-tools (not a direct dependency of this PR): https://github.com/opendatacube/odc-tools/pull/96

The build works locally, but is hitting an error in pytest collectoin within Github Actions. @omad mentioned similar problems have been  happening elsewhere too.

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
/usr/lib/python3.6/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:994: in _gcd_import
    ???
<frozen importlib._bootstrap>:971: in _find_and_load
    ???
<frozen importlib._bootstrap>:955: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:665: in _load_unlocked
    ???
/env/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:171: in exec_module
    exec(co, module.__dict__)
integration_tests/conftest.py:15: in <module>
    import cubedash
cubedash/__init__.py:6: in <module>
    from ._pages import app
cubedash/_pages.py:13: in <module>
    from cubedash import _audit, _monitoring
cubedash/_audit.py:14: in <module>
    from . import _model
cubedash/_model.py:17: in <module>
    from cubedash.summary import SummaryStore, TimePeriodOverview
cubedash/summary/__init__.py:3: in <module>
    from ._stores import SummaryStore
cubedash/summary/_stores.py:31: in <module>
    from sqlalchemy.engine import Engine, RowProxy
E   ImportError: cannot import name 'RowProxy'

@jeremyh I am using this PR to try and fix the CI : https://github.com/opendatacube/datacube-explorer/pull/210 expect something later today.

@jez are there DB schema chages required for this PR ?

@jez are there DB schema chages required for this PR ?

Nope! 🙂


Support Apple Silicon Build

https://github.com/dwavesystems/dimod/issues/868#issue-934633704

Codecov Report

Merging #875 (b84eff8) into main (06d2fd1) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #875   +/-   ##
=======================================
  Coverage   85.56%   85.56%
=======================================
  Files          75       75
  Lines        6554     6554
=======================================
  Hits         5608     5608
  Misses        946      946

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 06d2fd1...b84eff8. Read the comment docs.

https://github.com/dwavesystems/dimod/issues/870#issue-935320117

#1041 adds support for OSX universal wheels


No length validation on text fields

Description
On the Socials stage of the Application, I put in a value for Stack Exchange that was over 1 million characters. Not only was it not validated, it also seems to have saved correctly. This seems like an easy attack vector for somebody to overflow storage or raise storage billing by dropping in high-volume text directly into the database.
Expected behavior
Fields should have character length requirements and should be validated on the form client-side and server-side. Form should not advance and save extremely large values.
Steps To Reproduce

Go to '/Application'
Advance to a stage with a text field. I used Stack Exchange field on the Socials stage
Create an incredibly large value in something like Notepad++
Copy to the field.
Advance the form.
No error is shown, advances correctly.
Return back to the stage.
Value seems to be stored. Copy/pasting out to Notepad++ shows full value, not trimmed value.

Possible Fix
Field validator for length, along with updates to the labels to indicate limits to the user.

Hi @mrtamasvarga  - can you please let us know what you would like to limit the characters to? Once we get these details for the form, we can get this added in.

All single line text should be no more than 256 characters, and textarea/multi line texts should be limited to 2000 characters.

This is now closed as the repository is archived.
This has already been addressed in the new repository.


database silently failing, killing controller

Hi everyone. I discovered this issue a couple of days ago while not being able to register a new user with deis register. @gabrtv lent me a hand and we saw that controller was reporting:
-- Logs begin at Sat 2015-03-07 01:09:39 UTC. --
Mar 11 05:49:13 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:13Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:18 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:18Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:23 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:23Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:28 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:28Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:33 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:33Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:38 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:38Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:43 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:43Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:48 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:48Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:53 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:53Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:49:58 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:49:58Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:03 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:03Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:08 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:08Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:13 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:13Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:18 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:18Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:23 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:23Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:28 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:28Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:33 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:33Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:38 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:38Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:43 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:43Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:48 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:48Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:54 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:54Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:50:59 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:50:59Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:51:04 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:51:04Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:51:09 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:51:09Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:51:14 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:51:14Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:51:19 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:51:19Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1
Mar 11 05:51:24 dos-deis-coreos-4 sh[48192]: 2015-03-11T05:51:24Z 08c3e83b2fff confd[256]: ERROR Config check failed: exit status 1

We checked database and it stated, over and over again:
Mar 11 05:51:11 dos-deis-coreos-0 sh[52733]: HINT: A better error message should be written to handle this exception.  Please report this output and, if possible, the situation under which it arises.
Mar 11 05:51:11 dos-deis-coreos-0 sh[52733]: STRUCTURED: time=2015-03-11T05:51:11.955218-00 pid=33157
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: wal_e.retries WARNING  MSG: retrying after encountering exception
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: DETAIL: Exception information dump:
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: Traceback (most recent call last):
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/wal_e/retries.py", line 62, in shim
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: return f(*args, **kwargs)
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/wal_e/worker/s3/s3_deleter.py", line 27, in _delete_batch
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: bucket.delete_keys([key.name for key in page])
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/boto/s3/bucket.py", line 726, in delete_keys
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: while delete_keys2(headers):
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/boto/s3/bucket.py", line 714, in delete_keys2
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: data=data)
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 664, in make_request
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: retry_handler=retry_handler
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 1070, in make_request
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: retry_handler=retry_handler)
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 1029, in _mexe
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: raise ex
Mar 11 05:52:28 dos-deis-coreos-0 sh[52733]: timeout: timed out

After restarting both components the bug went away, but I'm seeing this happen every day now. Can anyone help me find a permanent solution?
Running on Azure, 5-box cluster, deis 1.4.1.
Here are my /opt/bin/deis-debug-logs

Is deis-store-gateway healthy? What do its logs look like? I wonder if #3183 will help with this...

I dunno. How do I check it?

Oh, sorry. I did ceph -s and it's ok

Gateway shouldn't emit many logs, so deisctl journal store-gateway@1 should show errors if there are any.
I'm wondering if gateway just becomes unresponsive, and that's why database can't write its WAL files anymore.

Oh, sorry. I did ceph -s and it's ok. The thing is I have a cron job to run every hour so my clocks don't go too astray. I'm on Azure and I get 1s+ offsets every couple of hours if not related: #3222

deisctl journal store-gateway@1 yields:
➜  app git:(development) deisctl journal store-gateway@1
^[f^[f-- Logs begin at Sat 2015-03-07 01:07:32 UTC. --
Mar 07 01:25:32 dos-deis-coreos-0 sh[2424]: curl: (52) Empty reply from server
Mar 07 01:25:33 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:33 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:34 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:34 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:35 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:35 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:36 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:36 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:37 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:37 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:38 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:38 dos-deis-coreos-0 sh[2424]: curl: (52) Empty reply from server
Mar 07 01:25:39 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:39 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:40 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:40 dos-deis-coreos-0 sh[2424]: curl: (52) Empty reply from server
Mar 07 01:25:41 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:41 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:42 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:42 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:43 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:43 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:44 dos-deis-coreos-0 sh[2423]: Starting RADOS gateway...
Mar 07 01:25:44 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:44 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:44 dos-deis-coreos-0 sh[2423]: Starting client.radosgw.gateway...
Mar 07 01:25:44 dos-deis-coreos-0 sh[2423]: 2015-03-07 01:25:44.697775 7f6bc60d8840  0 ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578), process radosgw, pid 930
Mar 07 01:25:44 dos-deis-coreos-0 sh[2423]: /usr/bin/radosgw is running.
Mar 07 01:25:44 dos-deis-coreos-0 sh[2423]: Starting Apache...
Mar 07 01:25:45 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:45 dos-deis-coreos-0 sh[2424]: curl: (56) Recv failure: Connection reset by peer
Mar 07 01:25:46 dos-deis-coreos-0 sh[2423]: deis-store-gateway running...
Mar 07 01:25:46 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:47 dos-deis-coreos-0 sh[2424]: Waiting for ceph gateway on 8888/tcp...
Mar 07 01:25:47 dos-deis-coreos-0 systemd[1]: Started deis-store-gateway.
Mar 08 05:28:12 dos-deis-coreos-0 sh[2423]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
Mar 08 05:28:15 dos-deis-coreos-0 sh[2423]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
Mar 09 02:42:56 dos-deis-coreos-0 sh[2423]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
Mar 09 03:04:18 dos-deis-coreos-0 sh[2423]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]

I'm also seeint that particular last line as the last entry of my (just restarted) controller:
Mar 11 05:54:55 dos-deis-coreos-4 sh[44385]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]

Yeah, that means etcd is having trouble, which typically points to platform issues :(

What is journalctl -fu etcd on that CoreOS host?

for host 0:
-- Logs begin at Sat 2015-03-07 01:07:32 UTC. --
Mar 11 04:36:46 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 04:36:46.788 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10062 events at index 5147224 completed
Mar 11 04:47:03 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 04:47:03.664 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10008 events at index 5157232 completed
Mar 11 04:57:24 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 04:57:24.656 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10033 events at index 5167265 completed
Mar 11 05:07:41 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 05:07:41.365 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10003 events at index 5177268 completed
Mar 11 05:18:04 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 05:18:04.732 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10061 events at index 5187329 completed
Mar 11 05:28:26 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 05:28:26.970 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10071 events at index 5197400 completed
Mar 11 05:38:46 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 05:38:46.368 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10027 events at index 5207427 completed
Mar 11 05:49:02 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 05:49:02.711 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10015 events at index 5217442 completed
Mar 11 05:59:17 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 05:59:17.358 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10015 events at index 5227457 completed
Mar 11 06:09:23 dos-deis-coreos-0 etcd[771]: [etcd] Mar 11 06:09:23.006 INFO      | 08387f2b90d343619ea24e286b31c3c3: snapshot of 10005 events at index 5237462 completed

There are two hosts that don't seem to yield a regular log for journalctl -fu etc:
core@dos-deis-coreos-3 ~ $ journalctl -fu etcd
-- Logs begin at Sat 2015-03-07 01:09:12 UTC. --
Mar 11 05:08:34 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:08:34.063 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10051 events at index 5178132 completed
Mar 11 05:18:55 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:18:55.663 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10007 events at index 5188139 completed
Mar 11 05:28:15 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:28:15.874 INFO      | 47b45a1043144511977bd62c6e1abfd1: warning: heartbeat time out peer="37e821517d374746bd93dda66b7a4320" missed=1 backoff="2s"
Mar 11 05:29:14 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:29:14.333 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10004 events at index 5198143 completed
Mar 11 05:39:32 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:39:32.963 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10095 events at index 5208238 completed
Mar 11 05:49:54 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:49:54.743 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10033 events at index 5218271 completed
Mar 11 05:54:58 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 05:54:58.249 INFO      | 47b45a1043144511977bd62c6e1abfd1: warning: heartbeat time out peer="39b9aef639994faa9d166a1983b36e72" missed=1 backoff="2s"
Mar 11 06:00:04 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 06:00:04.359 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10006 events at index 5228277 completed
Mar 11 06:10:10 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 06:10:10.994 INFO      | 47b45a1043144511977bd62c6e1abfd1: snapshot of 10012 events at index 5238289 completed
Mar 11 06:14:30 dos-deis-coreos-3 etcd[776]: [etcd] Mar 11 06:14:30.480 INFO      | 47b45a1043144511977bd62c6e1abfd1: warning: heartbeat time out peer="37e821517d374746bd93dda66b7a4320" missed=1 backoff="2s"

and
-- Logs begin at Sat 2015-03-07 01:09:39 UTC. --
Mar 11 05:06:45 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:06:45.991 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10015 events at index 5176347 completed
Mar 11 05:17:06 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:17:06.945 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10017 events at index 5186364 completed
Mar 11 05:27:26 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:27:26.376 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10064 events at index 5196428 completed
Mar 11 05:37:47 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:37:47.090 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10027 events at index 5206455 completed
Mar 11 05:48:03 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:48:03.378 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10052 events at index 5216507 completed
Mar 11 05:54:46 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:54:46.338 INFO      | 39b9aef639994faa9d166a1983b36e72: warning: heartbeat near election timeout: 2.2358992s
Mar 11 05:54:48 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:54:48.036 INFO      | 39b9aef639994faa9d166a1983b36e72: warning: heartbeat near election timeout: 1.6681972s
Mar 11 05:54:57 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:54:57.139 INFO      | 39b9aef639994faa9d166a1983b36e72: warning: heartbeat near election timeout: 3.1268797s
Mar 11 05:58:20 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 05:58:20.744 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10030 events at index 5226537 completed
Mar 11 06:08:28 dos-deis-coreos-4 etcd[778]: [etcd] Mar 11 06:08:28.501 INFO      | 39b9aef639994faa9d166a1983b36e72: snapshot of 10028 events at index 5236565 completed

Also related #3223

@carmstrong I'm seeing this again just after a few hours of resetting both components. Any leads?

@carmstrong is this related to #2741 ?

Any leads?

Unfortunately this has nothing to do with the components themselves, but rather the underlying platform. Having heartbeat timeouts at 3s is going to cause lots of problems. This sounds like an Azure performance issue, but our provision scripts are doing everything they can to prepare the system for this.
I would raise this issue with Microsoft.

@carmstrong ok, already sent all my gists and findings to the Microsoft team; thanks

Closing, as there's nothing actionable we can do here (and I think @gvilarino is on another platform now).


Fix on collapsible section clickable icon

Issue : COPDS-1394
The goal is to move the icon to the left of the clickable section

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Datagrid default filter's input field does not have associated label

Describe the bug
This is an issue with accessibility. We got a report from AccessLevel where they note that there is no label associated to the input field in the filter popover. When form fields do not have a programmatic (accessible) label, assistive technologies may incorrectly render the label or provide no label at all to users. When labels are not present or are incorrect, users of assistive technologies may not be able to complete a form.
Their recommendation is that developers can define form labels using the label element whenever an on-screen label is present. The label element's value is then explicitly associated with the relevant form field using the for and id attributes. When on-screen text is not available or can not programmatically be contained in a label, developers can use another method to provide an accessible name such as by using the title, aria-label, or aria-labelledby attribute to provide the label.
How to reproduce
Simply use clr-dg-column component with default filter.
Steps to reproduce the behavior:

Go to a view containing clr-dg-column (e.g. https://clarity.design/documentation/datagrid/built-in-filters)
Click on filter icon
Right-Click on the filter field inside the popup and choose Inspect.
Check its accessibility properties in the debugger:

Expected behavior
Input field inside the popup should have associated label or aria-label.
Versions
App

Angular: [9]
Node: [10.16.1]
Clarity: [3.0.0]

Device:

Type: [PC]
OS: [Win10]
Browser [chrome]
Version [81]

This has been reported with accessibility input by @gkhadK in https://github.com/vmware/clarity/issues/4664

Closing as duplicate.


parameter for file location

 Issue by robbla
Monday Nov 13, 2017 at 10:16 GMT
Originally opened as https://github.com/ballerina-lang/composer/issues/4583

Would it be possible to have a parameter for the file location that composer opens on by default?
Either as a startup parameter, part of the sh or bat file or even just in a settings or config file
setting this will allow a different view, pointing to a directory where .bal files are stored and make work faster

 Comment by pahans
Monday Nov 20, 2017 at 05:02 GMT

From next composer release onwards, you can pass --file param to composer script to open a file by default. We will work on starting up with a default directory too.

Composer is discontinued hence closing the issue.


Create site logo.

Creating site logo will create better branding and a more attractive website.

Check out the beta in the icon branch.

Done. Rolling this out with 0.6, 'cause why not.


Doc genration fails if rules are not orderd correctly

Currenlty the docs generator is dependend on the right order of rules.
This will break the docs generation:
    public function rules()
    {
        return [
            'started_at' => 'date|before:ended_at',
            'ended_at' => 'date',
        ];
    }

This works just fine:
    public function rules()
    {
        return [
            'ended_at' => 'date',
            'started_at' => 'date|before:ended_at',
        ];
    }

Similar problem here
            "scheduled_load" => "required|date",
            "scheduled_departure" => [
                "required",
                "date",
                "before:scheduled_arrival"
            ],
            "scheduled_arrival" => "required|date|after:scheduled_departure",
            "scheduled_unload" => "required|date|after:scheduled_arrival"

After I saw @tecbeast42  response I changed a little bit the logic to use only the after rule
            "scheduled_load" => "required|date",
            "scheduled_departure" => [
                "required",
                "date",
                "after:scheduled_load"
            ],
            "scheduled_arrival" => "required|date|after:scheduled_departure",
            "scheduled_unload" => "required|date|after:scheduled_arrival"

Getting rid of FormRequest parsing (#343), so this won't apply in v3.


First draft docs and README with landr

Use landr conventions for README and documentation.

Added ARCHITECTURE.md
Define balena.yml
Create logo.png -- just hexagonal logo
Create docs/images/cloudBlock-logo -- hexagonal logo plus project name

@balena-ci I self-certify!


refactor: file renames

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Make testwebapp gcloud deployable

update the testwebapp subproject so that it can easily be deployed with mvn gcloud:deploy, then create a test script or unit test that accesses the tests remotely.

The testwebapp is already easily deployable.    The KitchenSink unit test that calls the testwebapp is also easy to modify to call the remote instance, however most tests are currently failing due to the need for testing side-effects.   So some work is needed to review the tests and see which can be easily tested remotely.
Alternately, we could adapt the unit tests so that all checking is done remotely and the response object is the one that reports success or failure.  This would allow testing with a normal browser.

replaced by:

https://github.com/GoogleCloudPlatform/jetty-runtime/issues/7
https://github.com/GoogleCloudPlatform/jetty-runtime/issues/8
https://github.com/GoogleCloudPlatform/jetty-runtime/issues/9