[SPARK-41259][SQL] SparkSQLDriver Output schema and result string should be consistent

What changes were proposed in this pull request?

The spark-sql shell result output of Apache Spark is compatible with the hive, but the schema output is not processed.
Let's say have a case like the following with spark.sql.cli.print.header true:
spark-sql> create table tbl1 (id string) using parquet;
Response code
Time taken: 1.076 seconds

Before this PR:
spark-sql> show tables;
namespace	tableName	isTemporary
tbl1

After this PR:
spark-sql> show tables;
tableName
tbl1

Why are the changes needed?

Bugfix
Does this PR introduce any user-facing change?

Yes, if the user uses the spark-sql shell and enables spark.sql.cli.print.header, will see the values and the corresponding schema.
How was this patch tested?

Exist tests and new test.

@cloud-fan Please take a look if you have time, thanks.

Friendly ping @cloud-fan , Please take a look if you find a time, thanks

@yaooqinn this is a good point. If we are sure this is only for CLI display, not thriftserver protocol, I agree we don't need to follow Hive.

If we are sure this is only for CLI display,

Yes. hiveResultString is only used in spark-sql CLI. The thrift server-side always uses command output schema. Maybe this is the inconsistent issue we face internally

Yes. hiveResultString is added to ensure compatibility with hive output.
hiveResultString is only used by the spark-sql CLI. It is used only as the CLI display.
thriftServer always outputs as spark's schema,

hiveResultString is not used for the thrift protocol.
spark-sql output from the CLI and thriftServer is inconsistent.

I'm not sure why spark-sql CLI has to be compatible with hive output, personally, I don't think it's necessary. Maybe we should display spark's schema as is, just like thriftSever?

Please take a look, thanks @cloud-fan @yaooqinn

Seems like we just need a small change: only invoke hiveResultString if a testing config is true, and we only set that testing config in HiveComparisonTest.

Seems like we just need a small change: only invoke hiveResultString if a testing config is true, and we only set that testing config in HiveComparisonTest.

Return the result as a hive compatible sequence of strings of hiveResultString  currently does two things:

HiveComparisonTest used to compare compatibility with hive output;
Spark generates golden files for SQLQueryTestSuite;

2 is a friendly form of output, and I think we should keep it.

Spark generates golden files for SQLQueryTestSuite;

I think golden files there should match df.show instead of hive result.

Spark generates golden files for SQLQueryTestSuite;

I think golden files there should match df.show instead of hive result.

This may be a bit of a regression, maybe...  first, golden generation and validation are done with hiveResultString , it won't have what problem, then, the output is more readable, finally, ThriftServerQueryTestSuite test case generation also uses the independent hiveString, this also needs to reconstruct. So what's the benefit of removal?

It would be best if we had a separate 'df.show' method to format the results.

Like following:

And expression describes ExpressionInfo, it uses NULL instead of null

I'm looking for consistency. df.show is what users see, and hiveResultString is for golden files. Shouldn't the golden file match what users really see? Why do we test something that is almost invisible to users?

I'm looking for consistency. df.show is what users see, and hiveResultString is for golden files. Shouldn't the golden file match what users really see? Why do we test something that is almost invisible to users?

In fact, SQLQueryTestSuite does not test df.show, it is actually something under the DataSet, and hiveResultString only prints out the data of the DataSet, but I agree with you. It is better to print with spark's resultString

also cc @AngersZhuuuu

Adjusting df.show may need to change the output of show first. Some data values do not have a nice string representation yet

After code validation, ThriftServerQueryTestSuite and SQLQueryTestSuite depend on goldgen files;
If goldgen file follows the format of df.show (the format of df.show depends on the DF rowEncoder);
For SQLQueryTestSuite, you can easily generate df.show format;
For ThriftServerQueryTestSuite, as a result of this suite ThriftServer, based on the client side output as the Object, not the DataSet, It is difficult to match the format of df.show (unless you copy the logic of a DataSet encoder);
In addition to ThriftServerQueryTestSuite, it seems that it does not have to follow the df. The format of the show?
In both of these kits, which are internal tests, is the format of output based on the DataSet only required to be consistent, rather than having to use the format of df.show? Because it's not testing at df.show

Some data values do not have a nice string representation yet

Let's fix df.show first.

Hmm, how about add a legacy conf to indicate whether we should follow the hive format here? and default we use false.  I'm not sure if there will be users who use this place to connect to the internal system strictly in accordance with the hive format

Does ThriftServerQueryTestSuite also need to use sparkResultString golden file format?

@AngersZhuuuu are you programmatically parsing the output of the CLI? I thought it displays values for humans only.

https://github.com/apache/spark/pull/40699 is merged, can we continue this PR?

#40699 is merged, can we continue this PR?

Sure

https://github.com/apache/spark/pull/40922 has been merged, @Yikf can you open a PR to update ToPrettyString for all the needed changes to adjust the behavior of df.show?

#40922 has been merged, @Yikf can you open a PR to update ToPrettyString for all the needed changes to adjust the behavior of df.show?

Sure, I will open PR to make ToPrettyString eval nice strings.
And, I will close this PR since I think spark-sql and df.show are two different REPLs, don't have to make them exactly the same to break compatibility.


Fix possible mistakes in crew ordering

Following on from merge request bc1e214d65efc2652090508dfd29f98f77f1bdd9 and b7d260ac2336f7a96dfa8eb2d7aa78081ae63e89, I implemented support for checking that crews from a single club appear in numerical order.  I've since fixed a bunch of the CRA results, there are more errors flagged by CRA results but they all match with the charts on https://cantabsrowing.org.uk/, so are likely not mistakes.
The remaining errors flagged from Eights, Lents, Mays & Torpids should be checked, since they may point to more errors in crew listings.  See the text file attached.
wrong.txt

@johnwalley I've found more potential crew errors for us to check out here.

The college bumps errors feel like they're going to mostly be real errors int he crew listings but I suppose we should check.
The 1984 womens Torpids looks like New College II should be New College for example (looking at the multi year charts).

I have a CD, which I've mislaid, with Lents results on it. If I can find it, and find a way to read it, I can double check the Lents crew ordering.

Looking at charts that span multiple years, these errors make a little more sense.
1957: LMBC 5 bumped LMBC 4 on the last day in 1956, so the start ordering for 1957 is from the finish order.  In 1957 the bumps don't bump each other, in 1958 the numbers are swapped for the start order.  So either 1957 is a mistake or they were allowed to have boats out of order for whatever reason.
1966: Downing is similar, their 3rd boat bumps their 2nd boat the previous year, so the start order is the same as the finish order from 1965.  After 1966 the order was swapped back correctly for 1967.
1988: There are two Girton 2nd boats, starting next to each other.  Looking at the 1987/1988/1989 results in combination, I'm certain that the second of these is actually Homerton 2.  I've made that correction with 7e0eda5db4c7930912561a570baefcefc0bdd00d.
1989-1991: In 1988 LMBC 5 bumps LMBC 4, so the 1989 start order reflects the 1988 finish order.  This is maintained for the next couple of years, in 1992 the results have the crew orders corrected.
My guess is that the other issues are mistakes in the crew order, but we'd need other sources to check.  I've asked George Gilbert if he can help, since his book & CD are out of print.

I've just checked 1957 and 1966 and both are mistakes by me. I think it's safe to assume the others are mine too.
I have a copy of The Bumps; which is where I must have got these results and I've just checked against. I can provide a copy of the CD if it would be of use. Although, if we can get the raw results that would be super useful. There will almost certainly be other mistakes where I've manually entered results based on charts.

I've committed those changes to Lents Men's results in bee182d0ba51527e2ef0ed077ebcb12bbc1ddb56.

Lent's women results had one issue in 2012, Clare Hall were listed as their 2nd boat not their first boat, I've fixed this to match http://www.cucbc.org/bumps/results/7/2 in aa85d1fb79a52b4b233dda73da5d191170247e36.

Mays Womens:
2004: New Hall 4 and 5 swapped in the bottom division - this matches a bumps chart I have from this year.  Wikipedia only has top two divisions.  I'll leave this one was it is.
2005:  Peterhouse 3: the bumps chart I have from this year has their crew at the top of the last division as Peterhouse 2.,
Mays Men:
2010: LMBC 5 & 6: fixed to match https://cucbc.org/charts?year=2010&event=M&day=Fi&sex=M
2014: Clare 5: Clare 5 changed to Cl4 to match http://www.cucbc.org/bumps/results/12/3
2014: Pembroke 6: Left as is, looks like Pb5 failed to get on, matches http://www.cucbc.org/bumps/results/12/3
2015: Clare 4: Renamed Cl5 to Cl4 to match http://www.cucbc.org/bumps/results/14/1
2016: Caius 4: Renamed Ca5 to Ca4 to match https://www.cucbc.org/bumps/results/16/3
2017: Caius 4: Renamed Ca5 to Ca4 to match https://www.cucbc.org/bumps/results/18/3
2018: Fitz 3: Renamed F5 to F3 to match https://www.cucbc.org/bumps/results/20/3
Changes above committed as cf2322f6ce10d0c1470534a2e85d7fd2f91d3f24.

Torpids Women:
1984: New College 2: I agree that this should be New College, not their second boat, given the results over the year before and years after.
1994 & 1995: St John's 3: Nothing to help here from other years or sites other than Anu's, I'll leave as this is.
Torpids Men:
1984: Queens 3/4: These boats swapped over in 1982, then the lower boat drops out, in 1983 it's listed as Q3, then 1984 as Q4, then 1985 onwards as Q3.  I think Anu's results are wrong so I'll change this to Q3.
1999: SEH 2: First boat miss one year, but are then put back in a the following year, so I'll leave this as SEH2.
2001: Linacre 2: Same as above, first boat misses a year, then put back.  Leaving this as it is.
Committed these changes as 6a04d056c7c37b6fd7866a2e811ceb9ed6d5a3ca.

Updated list of incorrect crew number warnings:
wrong.txt
I've created #14 to help avoid printing this error when it's intentional, I think that's a better strategy than turning the error off.
I'll leave this issue open as I'll review each of the errors again before mark it as intentional.

I've reviewed the crew order warnings from Eights results, they match Anu's charts but those may have errors.
1958 Men - Oriel 5 (missing 4th boat): not enough data from previous/next year to say this is an error
1984 Women - New College 3 (missing 2nd boat): seem more like to be their second boat, based on the following 2 years, but it's not clear
1999 Women - Keble 4 (missing 3rd boat): no information from previous or next year to know if this is an error
2004 Men - Christ Church 3 & 4: given the results from previous & next year I'm certain that this is an error, so I've fixed this in a6f319eec581770f0e9d8dc42dc71229fdd9bdce.
2004 Women - Queen's 3 & 4 (4th above 3rd boat): No information from previous or next year to say this is an error


Validation loss goes to infinity

I have a large graph with 7000 nodes and 20 million triples with 110 class types and 59 relation types. Somewhere along the training, the validation loss suddenly becomes infinite. We also tried this on a much smaller graph with 7000 nodes and 117k triples with 110 class types and 10 relation types. The validation loss somehow kept increasing instead of decreasing until it went to infinity suddenly.
Any insights into what might be causing it and how to debug? Is it just an issue of the graph size or could something else be wrong?

I have not experienced this myself with this code release, have you maybe made some changes to the implementation?
Maybe it is also worth trying this PyTorch implementation: https://docs.dgl.ai/tutorials/models/1_gnn/4_rgcn.html

the infinity maybe related to the dataself


Shallow water solvers with bathymetry

This PR adds 3 solvers:

A 1D f-wave solver for SW with bathymetry;
A 2D f-wave solver for SW with bathymetry (normal only);
Normal and transverse versions of the "augmented" 2D SW solvers from Geoclaw.  These solvers are made PyClaw-compatible by avoiding the use of modules, which f2py couldn't handle.  In the process of making this work, I removed some functionality, like mappings to the sphere and alternative solver versions. This could all be added back in; I was just trying to minimize the number of places where bugs could occur.

This is a replacement for #86, which has been waiting for over 2 years.  I know the long-term dream is to do a major overhaul of the Geoclaw solvers, but I don't want to wait on that before merging this.

The augmented solver here is used in https://github.com/clawpack/apps/blob/master/notebooks/pyclaw/beach.ipynb.
The 1D f-wave solver is tested in https://github.com/clawpack/pyclaw/pull/564.

Is the Python version and Fortran version consistent?  I vaguely recall that there may be a discrepancy that should be fixed.

Also thanks for resurrecting this.  My dream of refactoring the solver was too ambitious clearly.

The Python and Fortran 1D f-wave solvers do give slightly different answers, so they must be doing something different.  Let me take a look.

I ran a quick test and they are pretty close:
https://gist.github.com/ketch/8adf1ba9eb6e11ca8dfa996e7e1193a5
They are a bit different but both seem "correct" to me.

I can take a look, I may remember what I thought might have been wrong.

so the big difference between the two is that Python uses HLL like speeds and the Fortran uses the Roe speeds.  We should probably both change to using the Einfeldt speeds.  I can make the changes to both if you want.

That would be great.  After you make the changes just issue a PR against my shallow_fwave branch.

I reran the comparison and now things match up to roundoff (check the gist linked above; updated).
I guess we should make the same changes to the 2D fwave solver...

Ah sorry, should have checked that.  I can do that as well.

So I have something done but I was not sure if there was a test for the 2d solver.   From your previous comments it does not appear so but I wanted to make sure.  I am not confident right now that what I have is going to work as it required some significant reworking of what @ketch wrote.

You're right about the lack of a test.  I'll write one and try both versions.

I think we should go ahead and merge this so that https://github.com/clawpack/pyclaw/pull/564 will pass tests.  Neither one will pass without the other but I think we can be confident now that together they will pass (and every solver has a test or an app now).

Oh, but it's probably worth deleting the commented-out code first?  Sorry, I should have thought about that before I merged the secondary PR.

Okay, I've been checking more carefully and I was wrong.  Commit d765c00a8ad318b3e37e34660713824da3e7ba0b broke the 2D Riemann solver (it quickly generates NaNs for the test case).  It's not obvious to me what is wrong with those changes.  The older version gives results that look reasonable.

At the very least the determination of beta is not consistent.  I am hoping to get back to this soon so I can try and take a look again.

I think the 1 and 0 arguments to merge are backwards, too.  But I switched them and still got NaNs.

I fixed some things and switched methods for dealing with the normal and transverse components.  There were some issues with the test case that I have corrected but things are still not working with bathymetry.  I will continue to look into it.

I believe this is ready to merge now.


[Custom feeds] Land on the pending feed page after creating a second feed

Reported in Slack here - check for more details
BUG: After adding a second card feed, you're supposed to land on the feed pending screen for the second feed. Instead you land back on the first feed. Shown in :thread:

I would like to help with this issue 🙂

no payments will be needed

Working on this today. I've noticed that we have no loading state like we just applied in the Expensify card when making the call to API - maybe we should add one? And Error handling?
https://github.com/user-attachments/assets/624213ef-c894-47ff-bffe-859fb74f1220

PR ready for review. https://github.com/Expensify/App/pull/51719
For now - just landing on a proper page, I can cover the loading button and error handling in another PR

Checked with @mountiny on this one - we will handle payments once the custom feeds project is done. @allgandalf could you work on the checklist? https://github.com/Expensify/App/issues/51567#issuecomment-2462149426

Yes, thanks! once the checklist is done, we can close


Your current API plan does not include access to this endpoint

I use the Scrape to get information within the following scrupt:
data = scrape(words=['chinaafrica', 'sinoafrica'], since="2021-01-01", until="2023-01-01", from_account=None, interval=1,
              headless=False, display_type="Latest", save_images=False, lang="en")

And I get the error information
{"errors":[{"message":"Your current API plan does not include access to this endpoint, please see https://developer.twitter.com/en/docs/twitter-api for more information","code":467}]}

Some parts of Twitter are down, API is one of it https://twitter.com/TwitterSupport/status/1632792942262747136?s=20


FEATURE REQUEST: DateTime Picker in Forms Component.

Issue type
I'm submitting a ...  (check one with "x")

[x] feature request
[ ] bug report
[ ] question about the decisions made in the repository

Issue description
Current behavior:
Currently there is no Date Time Widget in the existing ngx-admin. If we use the default HTML input of datetime, it is not available for all browsers. e.g Firefox etc
Expected behavior:
There should be a datetime form input in the forms component. And it must remain same for all browsers. Also, the format of value it returns must be same for all browsers.
Steps to reproduce:
You can check in the live demo in the forms section. There is no datetime input widget.
Related code:
You have already done this in the AngularJS version of the theme i-e blur-admin
Link to git repo:
https://github.com/akveo/blur-admin
Other information:
npm, node, OS, Browser
OS: Windows 10
Node v8.11
NPM v5.6
Supported Browser: Chrome, Edge, Opera
Not Supported Browser: Firefox, Internet Explorer, Safari

REF: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/datetime-local

You can add the components ngb-datepicker and ngb-timepicker  which ship with ng-bootstrap
I have forked the starter kit project which you can run to see it in action
note, ngx-admin already references ng-bootstrap however you have to upgrade to 3.2 - look at the changes from my commit to see how to do this.

Thanks @wallaceturner . Appreciate your effort


chore(cluster) head request to version negotiation

Summary
We do not use /version-handshake endpoint, just send a simple HEAD request to /v1/wrpc,
If it works then we use wrpc, else we fall back to websocket + json.
Full changelog

Add a new function head_request_version in  version_negotiation/init.lua
Change the behavior of functioin request_version_negotiation
Check the result in init_worker

this has the same issue that forced us to move version negotiation as a wRPC service and not as a HTTP request:  the websocket and http client libraries won't use the same connection, so there's no guarantee both will go to the same server.
PR #8775 implements the inline negotiation and does the equivalent of this PR: "try the wRPC connection and fall back to old protocol if it can't connect"

@javierguerragiraldez What is the advantage of "keeping the connection on the same backend server". It seems to be very fragile and I don't know why would anyone want to deploy their CP cluster this way.

@javierguerragiraldez What is the advantage of "keeping the connection on the same backend server".

Otherwise it requires shared state even before establishing the connection, as the CP has to carry the negotiation result from one event to another.  If there's no guarantee that it's the same CP node, it has to be in a database.
When it's part of the wRPC itself, on the other hand, it seamlessly activates the right services (and version) for the connection, without any convoluted state management.


[bug] Docker logs emit invalid JSON with nested double quotes and braces

Hey @mendhak, something messes with the escaping of the payloads and makes the logged JSON invalid. Here is the emitted log:
httpecho_1      | -----------------
httpecho_1      | { path: '/',
httpecho_1      |   headers:
httpecho_1      |    { host: 'httpecho:8888',
httpecho_1      |      'user-agent': 'Alertmanager/0.21.0',
httpecho_1      |      'content-length': '905',
httpecho_1      |      'content-type': 'application/json' },
httpecho_1      |   method: 'POST',
httpecho_1      |   body: '{"receiver":"generic","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"WhatEver","foo_bar_qux":"foo_moo_zoom","severity":"warning"},"annotations":{"description":"A Prometheus job has disappeared\\\\n  VALUE = 2\\\\n  LABELS: map[]","summary":"Prometheus job missing (instance )"},"startsAt":"2020-11-03T17:09:36.14925565Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://9e6d80bea9ef:9090/graph?g0.expr=vector%282%29+%3E+vector%281%29\\u0026g0.tab=1","fingerprint":"d57ff78af6ac95e8"}],"groupLabels":{"alertname":"WhatEver"},"commonLabels":{"alertname":"WhatEver","foo_bar_qux":"foo_moo_zoom","severity":"warning"},"commonAnnotations":{"description":"A Prometheus job has disappeared\\\\n  VALUE = 2\\\\n  LABELS: map[]","summary":"Prometheus job missing (instance )"},"externalURL":"http://1217896f2a1d:9093","version":"4","groupKey":"{}:{alertname=\\"WhatEver\\"}","truncatedAlerts":0}\n',
httpecho_1      |   cookies: undefined,
httpecho_1      |   fresh: false,
httpecho_1      |   hostname: 'httpecho',
httpecho_1      |   ip: '::ffff:<IP_ADDRESS>',
httpecho_1      |   ips: [],
httpecho_1      |   protocol: 'http',
httpecho_1      |   query: {},
httpecho_1      |   subdomains: [],
httpecho_1      |   xhr: false,
httpecho_1      |   os: { hostname: '1b549382c54e' },
httpecho_1      |   connection: { servername: undefined } }
httpecho_1      | ::ffff:<IP_ADDRESS> - - [03/Nov/2020:17:10:06 +0000] "POST / HTTP/1.1" 200 1451 "-" "Alertmanager/0.21.0"

If you go ahead and copy paste the payload body into a validator you will get an error.
Parse error on line 35:
...":"{}:{alertname=\\"WhatEver\\"}",  "tr
-----------------------^
Expecting 'EOF', '}', ':', ',', ']', got 'undefined'

Is this expected or fixable? Or is there another way to get the "real" payload? I want to use your docker image to get payloads for testing stuff
If you want to reproduce my setup simply run docker-compose up with the set of files attached in the zip. You will have to wait 2 minutes for the request to arrive
docker-compose.zip

Here is the payload I would expect:
{
  "receiver": "generic",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {
        "alertname": "WhatEver",
        "foo_bar_qux": "foo_moo_zoom",
        "severity": "warning"
      },
      "annotations": {
        "description": "A Prometheus job has disappeared\\n  VALUE = 2\\n  LABELS: map[]",
        "summary": "Prometheus job missing (instance )"
      },
      "startsAt": "2020-11-03T17:51:36.14925565Z",
      "endsAt": "0001-01-01T00:00:00Z",
      "generatorURL": "http://9e6d80bea9ef:9090/graph?g0.expr=vector%282%29+%3E+vector%281%29&g0.tab=1",
      "fingerprint": "d57ff78af6ac95e8"
    }
  ],
  "groupLabels": {
    "alertname": "WhatEver"
  },
  "commonLabels": {
    "alertname": "WhatEver",
    "foo_bar_qux": "foo_moo_zoom",
    "severity": "warning"
  },
  "commonAnnotations": {
    "description": "A Prometheus job has disappeared\\n  VALUE = 2\\n  LABELS: map[]",
    "summary": "Prometheus job missing (instance )"
  },
  "externalURL": "http://1217896f2a1d:9093",
  "version": "4",
  "groupKey": "{}:{alertname=\"WhatEver\"}",
  "truncatedAlerts": 0
}

I think if I'm understanding this correctly, what you're actually asking for is the body to be parsed as JSON you can look at, if the content type passed in is JSON.  So to escape it and show it as an object, just as part of the rest of the response.
Yes I think this is possible.  I wouldn't touch the body though - as that's reflecting what was passed in.
What I can do is: if the Content-Type is application/json, I can add a .json property which will contain the nested object.

Example output. Notice the "json" property.  The output still wasn't passing jsonlint.com validation and I realized, JSON requires quoted keys.  So notice that all the keys in the output are quoted now.
httpecho_1      | -----------------
httpecho_1      | {
httpecho_1      |     "path": "/",
httpecho_1      |     "headers": {
httpecho_1      |         "host": "httpecho:8888",
httpecho_1      |         "user-agent": "Alertmanager/0.21.0",
httpecho_1      |         "content-length": "905",
httpecho_1      |         "content-type": "application/json"
httpecho_1      |     },
httpecho_1      |     "method": "POST",
httpecho_1      |     "body": "{\"receiver\":\"generic\",\"status\":\"firing\",\"alerts\":[{\"status\":\"firing\",\"labels\":{\"alertname\":\"WhatEver\",\"foo_bar_qux\":\"foo_moo_zoom\",\"severity\":\"warning\"},\"annotations\":{\"description\":\"A Prometheus job has disappeared\\\\n  VALUE = 2\\\\n  LABELS: map[]\",\"summary\":\"Prometheus job missing (instance )\"},\"startsAt\":\"2020-11-03T20:48:06.14925565Z\",\"endsAt\":\"0001-01-01T00:00:00Z\",\"generatorURL\":\"http://a413d934596f:9090/graph?g0.expr=vector%282%29+%3E+vector%281%29\\u0026g0.tab=1\",\"fingerprint\":\"d57ff78af6ac95e8\"}],\"groupLabels\":{\"alertname\":\"WhatEver\"},\"commonLabels\":{\"alertname\":\"WhatEver\",\"foo_bar_qux\":\"foo_moo_zoom\",\"severity\":\"warning\"},\"commonAnnotations\":{\"description\":\"A Prometheus job has disappeared\\\\n  VALUE = 2\\\\n  LABELS: map[]\",\"summary\":\"Prometheus job missing (instance )\"},\"externalURL\":\"http://b0834199a15d:9093\",\"version\":\"4\",\"groupKey\":\"{}:{alertname=\\\"WhatEver\\\"}\",\"truncatedAlerts\":0}\n",
httpecho_1      |     "fresh": false,
httpecho_1      |     "hostname": "httpecho",
httpecho_1      |     "ip": "::ffff:<IP_ADDRESS>",
httpecho_1      |     "ips": [],
httpecho_1      |     "protocol": "http",
httpecho_1      |     "query": {},
httpecho_1      |     "subdomains": [],
httpecho_1      |     "xhr": false,
httpecho_1      |     "os": {
httpecho_1      |         "hostname": "a15bbc8033d1"
httpecho_1      |     },
httpecho_1      |     "connection": {},
httpecho_1      |     "json": {
httpecho_1      |         "receiver": "generic",
httpecho_1      |         "status": "firing",
httpecho_1      |         "alerts": [
httpecho_1      |             {
httpecho_1      |                 "status": "firing",
httpecho_1      |                 "labels": {
httpecho_1      |                     "alertname": "WhatEver",
httpecho_1      |                     "foo_bar_qux": "foo_moo_zoom",
httpecho_1      |                     "severity": "warning"
httpecho_1      |                 },
httpecho_1      |                 "annotations": {
httpecho_1      |                     "description": "A Prometheus job has disappeared\\n  VALUE = 2\\n  LABELS: map[]",
httpecho_1      |                     "summary": "Prometheus job missing (instance )"
httpecho_1      |                 },
httpecho_1      |                 "startsAt": "2020-11-03T20:48:06.14925565Z",
httpecho_1      |                 "endsAt": "0001-01-01T00:00:00Z",
httpecho_1      |                 "generatorURL": "http://a413d934596f:9090/graph?g0.expr=vector%282%29+%3E+vector%281%29&g0.tab=1",
httpecho_1      |                 "fingerprint": "d57ff78af6ac95e8"
httpecho_1      |             }
httpecho_1      |         ],
httpecho_1      |         "groupLabels": {
httpecho_1      |             "alertname": "WhatEver"
httpecho_1      |         },
httpecho_1      |         "commonLabels": {
httpecho_1      |             "alertname": "WhatEver",
httpecho_1      |             "foo_bar_qux": "foo_moo_zoom",
httpecho_1      |             "severity": "warning"
httpecho_1      |         },
httpecho_1      |         "commonAnnotations": {
httpecho_1      |             "description": "A Prometheus job has disappeared\\n  VALUE = 2\\n  LABELS: map[]",
httpecho_1      |             "summary": "Prometheus job missing (instance )"
httpecho_1      |         },
httpecho_1      |         "externalURL": "http://b0834199a15d:9093",
httpecho_1      |         "version": "4",
httpecho_1      |         "groupKey": "{}:{alertname=\"WhatEver\"}",
httpecho_1      |         "truncatedAlerts": 0
httpecho_1      |     }
httpecho_1      | }
httpecho_1      | ::ffff:<IP_ADDRESS> - - [03/Nov/2020:20:48:36 +0000] "POST / HTTP/1.1" 200 2655 "-" "Alertmanager/0.21.0"

Here is the commit.  https://github.com/mendhak/docker-http-https-echo/commit/dc3973d0884472083f4052914166bf4d64880c34
I've put a tagged image on Docker Hub.  @trallnag  Can you test out this image with tag:
mendhak/http-https-echo:issue-10-json-payload-output

If everything's OK I can merge it later.

I've tried it and it's indeed working perfectly, thanks a lot

Also learned something new, I assumed that this "JSON-view" is the same as the raw body of the request

I've merged it, so the :latest image has this change now. Cheers.


adds testing suite for nested binary var matrix functions

Summary
So @andrjohns in #2461 you were totally correct that I wasn't handling nested array structures correctly for var matrix. This fixes that by

adding overloads for apply_scalar_binary() for handling var_value<Matrix> types
adding an overload for as_array_or_scalar() that makes a 2d array from std::vector<std::vector<Scalar>> types
Adding functions to the test suite for testing binary vectorized functions (also with integer containers on the left and right hand side)

With the overload for as_array_or_scalar() this was actually pretty darn easy!
Tests
@andrjohns  can you check expect_ad_vectorized_matvar to make sure I'm catching all the specializations I need? I was a bit confused and wasn't totally sure if I caught them all. You can run the functions changed in the PR with
./runTests.py test/unit/math/mix/fun/bessel_first_kind_test.cpp \
test/unit/math/mix/fun/bessel_second_kind_test.cpp \
test/unit/math/mix/fun/beta_test.cpp \
test/unit/math/mix/fun/binary_log_loss_test.cpp \
test/unit/math/mix/fun/falling_factorial_test.cpp

Side Effects
Nope
Release notes
Adds nested vectorized functions for the new matrix type.
Checklist

[x] Math issue #1805

[x] Copyright holder: Steve Bronder
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[x] the basic tests are passing

unit tests pass (to run, use: ./runTests.py test/unit)
header checks pass, (make test-headers)
dependencies checks pass, (make test-math-dependencies)
docs build, (make doxygen)
code passes the built in C++ standards checks (make cpplint)

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

Great! I'll have a look on the weekend


python.pkgs.asyncssh: fix tests

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

cc @worldofpeace

I think more different errors now?

That's why this is WIP.

I know, I just wanted to document what the errors were now.

@dotlambda Since the author has already made changes that fixes this, and you confirmed that it in fact does - I think it would be safe to close this and just wait for the point release.

If you prefer that, we can also add a patch to the derivation right now.

I don't think we can patch it properly without using this commit also https://github.com/ronf/asyncssh/commit/e2b07c1c17f313b761e9bca70e8398e53d290dce#diff-6656057f99102816922876dbfdd92cf2

This'll do it I think
diff --git a/tests/test_auth_keys.py b/tests/test_auth_keys.py
index 1d625ef..72a49f7 100644
--- a/tests/test_auth_keys.py
+++ b/tests/test_auth_keys.py
@@ -13,13 +13,13 @@
 """Unit tests for matching against authorized_keys file"""

 import unittest
-from unittest.mock import patch

 import asyncssh

-from .util import TempDirTestCase, x509_available
+from .util import TempDirTestCase, patch_getnameinfo, x509_available

+@patch_getnameinfo
 class _TestAuthorizedKeys(TempDirTestCase):
     """Unit tests for auth_keys module"""

@@ -69,36 +69,22 @@ class _TestAuthorizedKeys(TempDirTestCase):
     def match_keys(self, tests, x509=False):
         """Match against authorized keys"""

-        def getnameinfo(sockaddr, flags):
-            """Mock reverse DNS lookup of client address"""
-
-            # pylint: disable=unused-argument
-
-            host, port = sockaddr
-
-            if host == '<IP_ADDRESS>':
-                return ('localhost', port)
-            else:
-                return sockaddr
-
-        with patch('socket.getnameinfo', getnameinfo):
-            for keys, matches in tests:
-                auth_keys = self.build_keys(keys, x509)
-                for (msg, keynum, client_addr,
-                     cert_principals, match) in matches:
-                    with self.subTest(msg, x509=x509):
-                        if x509:
-                            result, trusted_cert = auth_keys.validate_x509(
-                                self.imported_certlist[keynum], client_addr)
-                            if (trusted_cert and trusted_cert.subject !=
-                                    self.imported_certlist[keynum].subject):
-                                result = None
-                        else:
-                            result = auth_keys.validate(
-                                self.imported_keylist[keynum], client_addr,
-                                cert_principals, keynum == 1)
-
-                        self.assertEqual(result is not None, match)
+        for keys, matches in tests:
+            auth_keys = self.build_keys(keys, x509)
+            for (msg, keynum, client_addr, cert_principals, match) in matches:
+                with self.subTest(msg, x509=x509):
+                    if x509:
+                        result, trusted_cert = auth_keys.validate_x509(
+                            self.imported_certlist[keynum], client_addr)
+                        if (trusted_cert and trusted_cert.subject !=
+                                self.imported_certlist[keynum].subject):
+                            result = None
+                    else:
+                        result = auth_keys.validate(
+                            self.imported_keylist[keynum], client_addr,
+                            cert_principals, keynum == 1)
+
+                    self.assertEqual(result is not None, match)

     def test_matches(self):
         """Test authorized keys matching"""
diff --git a/tests/test_connection_auth.py b/tests/test_connection_auth.py
index 3da8a5b..7973093 100644
--- a/tests/test_connection_auth.py
+++ b/tests/test_connection_auth.py
@@ -23,8 +23,8 @@ from asyncssh.packet import String
 from asyncssh.public_key import CERT_TYPE_USER, CERT_TYPE_HOST

 from .server import Server, ServerTestCase
-from .util import asynctest, gss_available, patch_gss, make_certificate
-from .util import x509_available
+from .util import asynctest, gss_available, patch_getnameinfo, patch_gss
+from .util import make_certificate, x509_available

 class _FailValidateHostSSHServerConnection(asyncssh.SSHServerConnection):
@@ -455,6 +455,7 @@ class _TestGSSFQDN(ServerTestCase):
         yield from conn.wait_closed()

+@patch_getnameinfo
 class _TestHostBasedAuth(ServerTestCase):
     """Unit tests for host-based authentication"""

@@ -667,6 +668,7 @@ class _TestHostBasedAsyncServerAuth(_TestHostBasedAuth):
                                     client_username='user')

+@patch_getnameinfo
 class _TestLimitedHostBasedSignatureAlgs(ServerTestCase):
     """Unit tests for limited host key signature algorithms"""

diff --git a/tests/util.py b/tests/util.py
index 42bb596..4d92ec3 100644
--- a/tests/util.py
+++ b/tests/util.py
@@ -84,6 +84,24 @@ def asynctest35(func):
     return async_wrapper

+def patch_getnameinfo(cls):
+    """Decorator for patching socket.getnameinfo"""
+
+    def getnameinfo(sockaddr, flags):
+        """Mock reverse DNS lookup of client address"""
+
+        # pylint: disable=unused-argument
+
+        host, port = sockaddr
+
+        if host == '<IP_ADDRESS>':
+            return ('localhost', port)
+        else:
+            return sockaddr
+
+    return patch('socket.getnameinfo', getnameinfo)(cls)
+
+
 def patch_gss(cls):
     """Decorator for patching GSSAPI classes"""

@GrahamcOfBorg build python3.pkgs.asyncssh

@worldofpeace Looks good to you?

Success on x86_64-linux (full log)
Attempted: python3.pkgs.asyncssh
Partial log (click to expand)

/nix/store/wgk45x9aq7z17si3vm1hsh2h3n6z2n49-python3.6-asyncssh-1.13.1

Success on aarch64-linux (full log)
Attempted: python3.pkgs.asyncssh
Partial log (click to expand)

/nix/store/igi3750hnax86w3qsaiab0d45xj3axm8-python3.6-asyncssh-1.13.1

@dotlambda Looks nice. Thanks for the help.


Timing problem

Hey all:
I installed the SDx 2017.2 tool in Ubuntu and was able to generate CSIM and almost finished the FPGA build.
However, I got this error when run 'make' under sdsoc_build and I didn't see the bitstream and uImage were generated.
Appreciate any suggestions.
Finished 1st of 5 tasks (FPGA synthesis). Elapsed time: 00h 31m 01s.
.....
Finished 2nd of 5 tasks (FPGA logic optimization). Elapsed time: 00h 00m 40s.
.....................
Finished 3rd of 5 tasks (FPGA logic placement). Elapsed time: 00h 03m 18s.
.........................
Finished 4th of 5 tasks (FPGA routing). Elapsed time: 00h 04m 35s.

===>The following messages were generated while  Compiling (bitstream) opencl binary: bin Log file: /home/ubuntu/workspace/bnn_fpga/cpp/accel/sdsoc_build/_sds/p0/_vpl/ipi/ipiimpl/ipiimpl.runs/impl_1/runme.log  :
ERROR: [VPL-1] Design failed to meet timing
WARNING: [VPL 60-732] Link warning: There is no resource utilization data in DSA, utilization DRC is skipped
ERROR: [VPL 60-704] Integration error, problem implementing dynamic region, route_design ERROR
ERROR: [VPL 60-806] Failed to finish platform linker
ERROR: [SdsCompiler 83-5019] Exiting sds++ : Error when calling '/opt/Xilinx/SDx/2017.2/bin/vpl  --iprepo /home/ubuntu/workspace/bnn_fpga/cpp/accel/sdsoc_build/_sds/iprepo/repo  --iprepo /opt/Xilinx/SDx/2017.2/data/ip/xilinx  --platform /opt/Xilinx/SDx/2017.2/platforms/zed/zed.xpfm  --temp_dir /home/ubuntu/workspace/bnn_fpga/cpp/accel/sdsoc_build/_sds/p0  --output_dir /home/ubuntu/workspace/bnn_fpga/cpp/accel/sdsoc_build/_sds/p0/vpl  --input_file /home/ubuntu/workspace/bnn_fpga/cpp/accel/sdsoc_build/_sds/p0/.xsd/top.bd.tcl  --target hw   --save_temps  --kernels top --webtalk_flag SDSoC'
sds++ log file saved as /home/ubuntu/workspace/bnn_fpga/cpp/accel/sdsoc_build/_sds/reports/sds.log
ERROR: [SdsCompiler 83-5004] Build failed

Makefile:35: recipe for target 'accel_test_bnn.exe' failed
make: *** [accel_test_bnn.exe] Error 1

I've only tried the design with SDSoC, not SDx. I know these are different tools though I'm not sure what the differences are. Can you try compiling with SDSoC?

Thanks you again,  Ritchie.  We definitely should try SDDoC.

Closing due to lack of activity.


Feat: add alb-controller EKS component

what

Add alb-controller EKS component

why

alb-controller manages ALBs and NLBs based on Ingress and Service k8s annotations, and makes the transparent management of public-facing services via k8s objects trivial.

references

N/A

Two questions:

Can you get the pre-commit check to pass? This might involve installing pre-commit hooks locally.
Just to confirm, have you deployed this component for a customer already?


[webview_flutter]: add option to enable DOM storage in android webview

Description
As we all known, like JavaScript, android webview is disable DOM storage by default.
To enable it, this PR provides an option to enable DOM storage in android webview.
Related Issues
flutter/flutter#26133
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
[x] All existing and new tests are passing.
[x] I updated/added relevant documentation (doc comments with ///).
[x] The analyzer (flutter analyze) does not report any problems on my PR.
[x] I read and followed the Flutter Style Guide.
[x] The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
[x] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
[x] I updated CHANGELOG.md to add a description of the change.
[x] I signed the CLA.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
[x] No, this is not a breaking change.

Thanks for sending this!
I started a discussion regarding what kind of API we want to enable the web storage https://github.com/flutter/flutter/issues/26133#issuecomment-510570085 I think we should conclude that discussion before moving forward with this PR.
I'm labeling this PR with the backlog label until the API discussion is concluded and the issue is prioritized.
I don't want to make you do extra work before we know we're going to land this(so probably worth waiting for the API discussion to conclude), but one thing I'd like to have before landing this PR is an e2e test(e.g what we have here.

please close issue and merge this functionality in release


Release v1.12.0

What's Changed
New Features 🎉

feat(theme): add afterDocFooter slot for custom theme @sanyuan0704  #646
feat(theme): add scroll to top button through enableScrollToTop config @ArnavK-09  #622
feat(theme): pass through HTMLAnchorElement props to Link component. @sumy7  #641
feat(theme): add DocLayout export @sanyuan0704  #635
feat(search): set min time of initing search index as 1s @sanyuan0704  #647
feat(icons): new social platform for social links——X  @ArnavK-09  #618
feat(core): trigger auto page reload after page meta changed @sanyuan0704  #640

Bug Fixes 🐞

fix(doc-components): change code children to string @10Derozan  #642
fix(theme-default): fix incorrect css variable name @inottn  #606
fix(plugin-preview): support rspress restart @10Derozan  #615
fix(theme-default): private import not work in production @sanyuan0704 #637
fix(theme): use logical OR instead of nullish coalescing when label is empty @10Derozan  #605
fix(core): update sidebar after h1 change @sanyuan0704  #640
fix(core): doc fragments' headers disappeared in development @sanyuan0704  #640
fix(theme):overflown tabs not showing on mobile fixed @ArnavK-09  #616

Other Changes

chore(theme-default): adjust copyCodeBtn icon size & updated wordWrapBtn animations for equality between both buttons @ArnavK-09  #613
refactor(theme-default): ui switch query @sanyuan0704  #634

🎉


Reuse heartbeat's msg in order to save disk space

Noticed that some common messages are keep repeating and I believe the database will not compress them.

Is this closed by mistake?

Is this closed by mistake?

Sorry about the extra notifications and good catch for noticing this mistake.
I did delete all branches which had been merged earlier.
I likely should have been more careful in this 😅
This branch was closed as the 2.0.X branch has been merged into master earlier.
I am currently going through and fixing this

Ops, I just fell into the same trap too. How did you reopen this?

😂😂 Maybe you have to restore the branch first, then change the target branch to something else?

,

😂😂 Maybe you have to restore the branch first, then change the target branch to something else?

I restored the branch.
Rebasing is something that I'd like to leave up to @louislam because maybe you have uncommitted changes/... (likely overparanoid, I know)


Requireish

@kamekazemaster have a read through this - it's based on your changes, but I'm trying to read the "requireish" from the config instead of forcing it to be specified programatically.  I feel quite sure that I'm missing something here.  -_-  But see what you think.  :)

This breaks a little bit in that you can no longer call "setConfig()", but I'm happy to deprecate that and bump the major version.  I'm happy to remove "configure()" for that matter, now that you can just pass configuration directly to a transform.  :P

I just have thought about the same alternative way to implement it like you did :D
I had the idea to make it like for example babelify which loads a .babelrc file as config. Alternatively (or additionally?) pass in configuration alá .transform(aliasify.configure({...}).
Some notices to your implementation:
I think you should merge the two configs passed in in the case where you call configure() and have a config file. The passed in one through .configure() with higher priority.
I don't completely understand how transformTools.loadTransformConfigSync does find your actual config file. Did I get it right that you have to set the path to your config file in the package.json? Wouldn't it be better to determine a static name required for the config file? (like babel's .babelrc file)
Another thing is that you don't pass requireish strings which should additionally be aliased instead you have to state all of them. I think your solution is better but this have to be changed in the doc also ;)

I think you should merge the two configs passed in in the case where you call configure() and have a config file. The passed in one through .configure() with higher priority.

We actually will end up merging them - you'll note we only extract requireish here and ignore everything else.  We also load it from config if there is a requireish in config, and otherwise we go load from package.json, even if config was passed in (so if both config and package.json have a requireish, we take the config one.)  But then at the end, we pass config into wrappedTransform, and down in browserify-transform-tools, it will merge them before we get into the real stuff.

I don't completely understand how transformTools.loadTransformConfigSync does find your actual config file. Did I get it right that you have to set the path to your config file in the package.json? Wouldn't it be better to determine a static name required for the config file? (like babel's .babelrc file)

I've never been a huge fan of loading out of package.json, but I stole it from somewhere else - I think from thlorenz/browserify-shim?  When I wrote browserify-transform-tools, there was no nice way to pass parameters to a transform, and substack seemed to initially be against the idea of passing them as a second parameter to the transform function (like they are now.)  That's by far the cleanest way to do it, though - you can do it programatically, and you can even pass them on the command line now:
browserify -t [ aliasify --requireish proxyquire ]

for example (I have no idea if I can specify requireish more than once to make an array - I should figure that out and add an example to the README.md.  :) )
I'd be perfectly happy to deprecate loading from package.json from browserify-transform-tools, but it's a "thing" now, and people do it, and I'd break everything.  :P
To answer your question more directly - you can either put the config straight into package.json, so have a section in package.json that looks like:
aliasify: {
    requireish: ['proxyquire']
    ...
}

Or you can specify a path to a JS config file (which is nicer, because then you can use variables and such.)

Another thing is that you don't pass requireish strings which should additionally be aliased instead you have to state all of them. I think your solution is better but this have to be changed in the doc also ;)

Good point.  I will fix this.  :)

TODO:

[ ] Document that you need to explicitly pass require in requireish if you want it to do what you think it should do.
[ ] Work out the best way to pass an array of values from the browserify command line, and document this.

If you want to read more about how browserify-transform-tools loads configuration, there's a lot of details here: https://github.com/benbria/browserify-transform-tools/wiki/Transform-Configuration

I guess there's not really a great way to pass a list of aliases on the command line.    But, for future reference, if you do:
browserify -t [ aliasify --requireish proxyquire ]

then aliasify will get passed a config of 'proxyquire', and if you do:
browserify -t [ aliasify --requireish proxyquire --requireish foo ]

then aliasify will get passed a config of ['proxyquire', 'foo'].  So we're halfway there.  :)  But we'd need to do something like: --alias foo:bar to pass aliases, which is out of scope of this PR for sure.  :)

Published as v2.0.0.  Sorry again this took so long.  Let me know if you run into any problems.  :)


Centos error

Error under CRON only :
/usr/local/vesta/bin/v-sync-backups-to-mega: line 44: megals: command not found

Most likely you have not installed the mega-tools, the automatic installation script only works for debian\ubuntu. Also if you have installed mega-tools probably not in your $PATH.

I installed as in https://www.geekytuts.net/shell-tricks/archivebackup-server-mega-megatools-centos-7/
All works from CLI but not from CRON.

On behalf of the user "root" run the following command
export PATH=$PATH:/usr/local/bin

Also it would be good to check the compiled files "megatools" command
ll /usr/local/bin/mega*

Result
-rwxr-xr-x 1 root root 380007 Sep  5 11:47 /usr/local/bin/megacopy
-rwxr-xr-x 1 root root 373534 Sep  5 11:47 /usr/local/bin/megadf
-rwxr-xr-x 1 root root 385271 Sep  5 11:47 /usr/local/bin/megadl
-rwxr-xr-x 1 root root 374191 Sep  5 11:47 /usr/local/bin/megaget
-rwxr-xr-x 1 root root 375618 Sep  5 11:47 /usr/local/bin/megals
-rwxr-xr-x 1 root root 372128 Sep  5 11:47 /usr/local/bin/megamkdir
-rwxr-xr-x 1 root root 373816 Sep  5 11:47 /usr/local/bin/megaput
-rwxr-xr-x 1 root root 377504 Sep  5 11:47 /usr/local/bin/megareg
-rwxr-xr-x 1 root root 372139 Sep  5 11:47 /usr/local/bin/megarm

Yes. It works! Thank you.

Hi! I have just the same problem. Centos 7 error under cron.
#------------------------#-------------------------#------------------------#
#------------------------|2018-03-21 19:21:01 +0100|------------------------#
#------------------------#-------------------------#------------------------#
[2018-03-21 19:21:01 +0100] -- Start script: #v-sync-backups-to-mega# backups upload to mega.
[2018-03-21 19:21:01 +0100] -- Run user: root - Params: 21 200000
[2018-03-21 19:21:01 +0100] -- Print users list: admin backup centos
[2018-03-21 19:21:01 +0100] -- Count backups upload: 21
[2018-03-21 19:21:01 +0100] -- Maximum number of rows in a log file: 200000
/usr/local/vesta/bin/v-sync-backups-to-mega: línea 44: megamkdir: command not found
[2018-03-21 19:21:01 +0100]
[2018-03-21 19:21:01 +0100] -- User log block: admin
[2018-03-21 19:21:01 +0100] -- Start remove admin cloud backups:
[2018-03-21 19:21:01 +0100] -- Stop remove admin cloud backups
[2018-03-21 19:21:01 +0100] -- Start upload admin backups:
[2018-03-21 19:21:01 +0100] -- Upload: /backup/admin.2018-03-21_05-14-44.tar
/usr/local/vesta/bin/v-sync-backups-to-mega: línea 65: megaput: command not found
[2018-03-21 19:21:01 +0100] -- Stop upload admin backups
[2018-03-21 19:21:01 +0100]
[2018-03-21 19:21:01 +0100] -- User log block: backup
[2018-03-21 19:21:01 +0100] -- Start remove backup cloud backups:
[2018-03-21 19:21:01 +0100] -- Stop remove backup cloud backups
[2018-03-21 19:21:01 +0100] -- Start upload backup backups:
[2018-03-21 19:21:01 +0100] -- Stop upload backup backups
[2018-03-21 19:21:01 +0100]
[2018-03-21 19:21:01 +0100] -- User log block: centos
In ubuntu was working fine.

Yes Vasily! if u add
export PATH=$PATH:/usr/local/bin
AFTER "#!/bin/bash" in ur script, WORKS PERFECT! THX

Happy for you!  I Added fixes to the repository.


CUJ Testing for Beta v2 - Creator Can Go to Their Settings Page

Feature Description
In preparation for the Beta 2 release we would like to have validation and confirmation of the CUJ functionality detailed in
https://github.com/google/web-stories-wp/issues/1673

[ ] Set GA tracking ID
[ ]  Upload and set a publisher logo globally
[ ]  Manage their logo

Settings section is out-of-scope - currently behind feature flag


Transaction list only shows last 100 transactions

The interface only fetches the last 100 incoming and outgoing transactions and the rest is truncated (because of pagination, which is not implemented in the react interface).
#431 can/must be solved along with this issue

this is related to #431

count_unique_nonce field has been added to the /transactions/ endpoint, it's already on every environment

Hi guys I need your opinion regarding this issue.
As discussed with @Uxio0 and @fernandomg, doing the implementation of the pagination on the frontend with the current endpoint's will give us problems on the frontend:

Now there are two endpoints one for transactions and another one for incoming transactions but we show all the transactions in one tab, so, in order to paginate, we would need to fetch both endpoints, reorder them and discard the ones that don't "enter" into the page. This makes that the pagination is just an illusion, there is no real pagination. We have several issues with the performance of the app, doing this won't improve it, maybe will downgrade it more.

Also force us to keep a register of which transactions were discarded in the last page to add them in the new page every time the users moves between pages

I think that the pagination should be solved by the backend in only one endpoint that provide us the params for fetching the transactions directly.
As discussed with @Uxio0 he said that doing this will cause problems on the backend because there are filters that aren't available for both types of transactions. My suggestion was to create a new endpoint just for this functionality
We have also discussed in the idea of dividing the transactions tab into two tabs: one for each type of transaction, doing this will directly solve the problem and the pagination will be easier.  BUT we need opinions because this will affect the user experience (@lukasschor), we think that the user will expect to have all the transactions merged into one tab, like the transactions of a normal bank account.
Please feel free to add comments and opinions !
CC: @rmeissner @pablofullana @mikheevm

Hi @Agupane!

Now there are two endpoints one for transactions and another one for incoming transactions but we show all the transactions in one tab, so, in order to paginate, we would need to fetch both endpoints, reorder them and discard the ones that don't "enter" into the page. This makes that the pagination is just an illusion, there is no real pagination. We have several issues with the performance of the app, doing this won't improve it, maybe will downgrade it more.

Could you explain the performance issues you are talking about? From my opinion the biggest impact would be not processing and generating the list every time that's retrieved from the service if the list does not change. Endpoints already support weak Etag. That's something that can be already done in frontend with the current implementation.

Also force us to keep a register of which transactions were discarded in the last page to add them in the new page every time the users moves between pages

I agree on this, pagination is not easy. The time filters made it easier, but as you say, it's something difficult to manage from the frontend.

I think that the pagination should be solved by the backend in only one endpoint that provide us the params for fetching the transactions directly.
As discussed with @Uxio0 he said that doing this will cause problems on the backend because there are filters that aren't available for both types of transactions. My suggestion was to create a new endpoint just for this functionality

Not just that. I don't want to have different objects in the same endpoint breaking typed clients, so I would need a proposal of an endpoint were every object is the same (with the same fields, even if some are null for some cases, e.g. incoming ether transfers will not have confirmations). It will not be easy also to do that in the backend as it will involve merging completely different tables, I'd be happy to implement that if we can think on an endpoint with all the use cases and the filters that will be needed in that case.

We have also discussed in the idea of dividing the transactions tab into two tabs: one for each type of transaction, doing this will directly solve the problem and the pagination will be easier. BUT we need opinions because this will affect the user experience (@lukasschor), we think that the user will expect to have all the transactions merged into one tab, like the transactions of a normal bank account.

I, as a user of the old multisig would like to have this feature, as I don't want the incoming txs messing with the transactions I'm preparing, but a filter will work too. Regular banks don't have "New owner", "You changed your password", "You are approving a new credit card" in the middle of the regular money transfers, but I understand that some users will expect to see all the transfers in the same tab.

Could you explain the performance issues you are talking about? From my opinion the biggest impact would be not processing and generating the list every time that's retrieved from the service if the list does not change. Endpoints already support weak Etag. That's something that can be already done in frontend with the current implementation.

Yep I agree with you but I also think that we would need to make two requests instead of one, now maybe it's not a big problem, but if we go mobile it will be a problem for slow connections, lets say we fetch a page of 100 txs, we will need to make two request and we will receive 100 txs from both endpoints than we need to reorder/discard unused to generate the list, instead of fetching directly the 100 transactions that we need
Usings etags would be a really good approach to improve the performance, should be implemented on the frontend of course :) but the problem of fetching twice the data at least the first time, will be still there

Not just that. I don't want to have different objects in the same endpoint breaking typed clients, so I would need a proposal of an endpoint were every object is the same (with the same fields, even if some are null for some cases, e.g. incoming ether transfers will not have confirmations). It will not be easy also to do that in the backend as it will involve merging completely different tables, I'd be happy to implement that if we can think on an endpoint with all the use cases and the filters that will be needed in that case.

A merge of both object would be what I expect, making just null the fields which are not present in one transaction, for instance:
Now:
Incoming transaction: { executionDate: '01-01-2020', blockNumber: 1, transactionHash: 0x..123, to: 0x..123, value: 5, tokenAddress: 0x..456, from: 0x...456}

Other Transaction: { executionDate: '01-01-2020', blockNumber: 1, transactionHash: 0x..123, to: 0x..123, value: 5,  data: 0x...789, gasToken: 0x...123,
operation: 0, confirmations: [], ......}

Merged obj from "transaction": { executionDate: '01-01-2020', blockNumber: 1, transactionHash: 0x..123, to: 0x..123, value: 5,  , tokenAddress:null from: null, data: 0x...789, gasToken: 0x...123,
operation: 0, confirmations: [], ......}

Merged obj from incoming transaction: { executionDate: '01-01-2020', blockNumber: 1, transactionHash: 0x..123, to: 0x..123, value: 5, tokenAddress: 0x..456, from: 0x...456, data: 0x...789, gasToken: 0x...123,
operation: null, confirmations: null, ......}

Let me know if I was clear enough :)

I, as a user of the old multisig would like to have this feature, as I don't want the incoming txs messing with the transactions I'm preparing, but a filter will work too. Regular banks don't have "New owner", "You changed your password", "You are approving a new credit card" in the middle of the regular money transfers, but I understand that some users will expect to see all the transfers in the same tab.

I agree with you, maybe we would need more feedback from the users regarding this

@rmeissner, @mikheevm, any thoughts on this?

I agree with Agus on this. It's very difficult to merge these two lists together given not any transaction is rendered, we need to sort them by date submitted, etc.
Also this will include further complexity when we add sorting, filters, etc
I would say the best solution (from frontend perspective) is to merge these two endpoints into one

@rmeissner @Uxio0 Any further thoughts on this?

I do agree that 1 endpoint makes it easier on the UI (always did). But we should properly define the endpoint and what we expect. Just saying "merge all properties and null the rest" is for me bad practise. Then we should define a proper polymorphic endpoint (e.g. add a type parameter with incoming_erc20, incoming_erc721 or outgoing_tx) this way we can add more potential transaction types in the future (e.g. incoming_erc1155 or maybe outgoing_ercX).
Also in general it is not that hard to merge these to lists with the current sorting (timestamp) ... it takes 2 requests and one time iterating through these lists.
Currently the backend doesn't apply any more magic (and this is currently not planned) -> e.g. filtering or grouping cancelation transactions. If this is a requirement please not them down clearly (e.g. specify what is a cancelation transaction).
Also we are kind of promoting our backend to be used be everyone, so having very specialized views for the clients are kind of weird. A solution could be to have a client gateway (basically a separate service in between the interface and the gnosis services).

Also in general it is not that hard to merge these to lists with the current sorting (timestamp) ... it takes 2 requests and one time iterating through these lists.

It's not about merging list into one, we already do this, this is about writing all the logic to get actually 100 transactions and further, keep in mind we also have cancellation transactions which are not shown as a separate transaction but only in the details (I guess you don't want them to count for a separate one)

we should properly define the endpoint and what we expect. Just saying "merge all properties and null the rest" is for me bad practise.

I don't think this is a problem at all, in the end they're all transactions on the ethereum blockchain

keep in mind we also have cancellation transactions which are not shown as a separate transaction

In this case the backend issue should be "Add possibility to filter cancellation transactions" ;)

I don't think this is a problem at all, in the end they're all transactions on the ethereum blockchain

This is bad API design because currently they have all different properties, as the incoming transfers are not really parsed transactions, but parsed events (so there is even a difference on the very low level)

In this case the backend issue should be "Add possibility to filter cancellation transactions" ;)

Yeah, if cancellation transaction would be treated as confirmation with type "REJECTION", it would solve this issue too. Currently we have only EXECUTION and CONFIRMATION or something like that, and Uxio told me that it's going to get removed 😃

Yeah, if cancellation transaction would be treated as confirmation with type "REJECTION", it would solve this issue too. Currently we have only EXECUTION and CONFIRMATION or something like that, and Uxio told me that it's going to get removed 😃

No, currently there's only ConfirmationType=CONFIRMATION inside every confirmation (that's something inherited from the alpha of the tx service when working with Adolfo), so you are not getting any information from that :)

guys, any agreement on how to proceed?

updating the backend by adding a new endpoint returning both incoming and outgoing transactions merged in one list. without breaking current API design in terms of already existing endpoints.
merging both lists on the frontend and dealing with complex and low-performing logic from now on.
displaying two different lists in the UI instead of just one.

still open for discussion

guys, any agreement on how to proceed?

updating the backend by adding a new endpoint returning both incoming and outgoing transactions merged in one list. without breaking current API design in terms of already existing endpoints.
merging both lists on the frontend and dealing with complex and low-performing logic from now on.
displaying two different lists in the UI instead of just one.

still open for discussion

Hi Pablo, we are going for the 1, but we are still deciding how to do it as it's not trivial

This depends on this backend ticket: https://github.com/gnosis/safe-transaction-service/issues/81
Putting it in the icebox for now.


Error on drag/drop into hp fields

This has not always been the case, but since the last update or two, this error has started popping up.
I often use the modifier box to fix rolls, such as when a player forgot to upcast and adds a normal die or when some modifier was forgotten, using a negative number to heal or positive number to deal damage.  In previous versions of CA (and without CA at all), I could just put this number in the lower-left modifier box and drag it onto the wounds or current hp field of the target (either their sheet or on the CT), and it would report a chat message of dealing damage or healing and take effect as expected.  Now, however, it seems that any drop onto the field is treated as a "set" instead of "adjust by" and I get a console error:
[ERROR]  Script execution error: [string "utility/scripts/dynamic_hp.lua"]:45: attempt to call field 'getActorFromCT' (a nil value)

This occurs when I have only CA v2.1.1 loaded with no other extensions. Attached is a mkv if you want to see it in action.
issue.zip

Should be working again in v2.1.2. Sorry for the inconvenience. And thanks for providing so thorough information!

Yes, this is working perfectly now. Thanks!


No support for RecursiveDo extension?

I have installed:
hlint-1.8.61
Configuring haskell-src-exts-<IP_ADDRESS>
What I do:
{-# LANGUAGE RecursiveDo #-}

test = do
    rec x <- getLine

    return x

What I supposed to get:

ghc compiles;
hlint - no complain.

What do I get:

ghc compiles;
hlint - not:

Warning: Parse error in pattern: rec
Found:
    test = do
  >     rec x <- getLine

        return x

I use hlint as a part of SublimeHaskell and I have no clue how to pass -XArrows to it.

I'm getting the same error, different context. GHC doesn't complain. I found rec used as a normal identifier for a function, and I still get the error message when just typing rec into a new .hs file and waiting half a second for my editor plugin to execute hlint. The issue on the haskell-src-exts project is closed, so I'm unsure if I should have commented there instead.

@jplatte The problem is that HLint as standard turns on as many extensions as it can that don't steal "too much" syntax. In your case, the DoRec extension steals the rec identifier as a keyword. I would generally discourage using rec as an identifier because it is a keyword under some extensions, and it is also used to indicate recursive let bindings in the Core language, so it has great scope for confusion. If you do want to continue to use rec as a keyword, passing -XNoDoRec will disable the conflicting extension, and then your program will parse.

As @jplatte mentioned, the ticket was resolved upstream, and I've just tested the original report, and it works fine now, so closing this ticket.

Alright, thanks for the info. I didn't write the code with the rec identifier in it, but it's a project I've started working on recently (haskell-gi). I'll ask the original author about that identifier, but I think it shouldn't be a problem to change it.


Horizontal split view for Filesystem Dock

Seems fine now. One concern is that when you switch between horizontal and vertical mode, the split will be wrong. Maybe we should keep it separately for 2 modes

Check now please

The split seems to be not remembered between sessions.
Set vertical split size and horizontal split size. Restart while the dock is in horizontal mode. The horizontal split will reset, while the vertical split will use the horizontal value. I didn't check the implementation, but they need to be stored in separate values.

Thanks! And congrats for your first merged Godot contribution :tada:


Page updates

Hi,
Is this page going to be updated? The links are broken as well.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d7f34ef7-cce5-a11e-6e43-87bd1ea844c6
Version Independent ID: 125a13e2-c2bb-0ca2-322a-66c88e695423
Content: Pipelines REST API for Azure DevOps Services - Azure DevOps Services REST API
Content Source: docs-ref-conceptual/5.1/distributedTask/index.md
Product: devops
Technology: devops-ref
GitHub Login: @wnjenkin
Microsoft Alias: elbatk

+1
If the page is not going to be updated, any place where we could find updated documentation?

Hi @ginakkiim ,
Just in case it helps, this is the swagger (v6.0) of the related section.


Support custom RTP header extension

Fix https://github.com/livekit/livekit/issues/3111
Key updates:

Implemented configurable custom RTP header extensions.
Enhanced SFU forwarding to transmit Publisher's header extensions to Subscriber.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Inconsistent data response on semantic admin endpoint.

If you make a request to the semantic admin as such:
/local_server/server_admin/json.php?admin_action=get_permissions
And the user you're logging in as has only one service body the response will not be an array.  However if there are more than one it will be.   It's my belief that it should always return a consistent model (in this case, always return an array).  I realize this might break other clients and things will have to be adjusted accordingly.  I will be working around this issue in the interim to not block development on this issue found: https://github.com/radius314/yap/issues/89
One service body (no array):

More than one service body (array):

You are correct. There are probably a number of things like this throughout the repo.
The issue is that fixing them (like the gKey issue) may break stuff that is out there.
It will take time to analyze this, and time is something that's in short supply for me, these days.

Worked around it, so it’s non-urgent.

On Aug 3, 2018, at 8:40 AM, Chris Marshall<EMAIL_ADDRESS>wrote:
You are correct. There are probably a number of things like this throughout the repo.
The issue is that fixing them (like the gKey issue) may break stuff that is out there.
It will take time to analyze this, and time is something that's in short supply for me, these days.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/LittleGreenViper/BMLT-Root-Server/issues/12#issuecomment-410241123, or mute the thread https://github.com/notifications/unsubscribe-auth/ABw9u4_OX2glCDUGxysjscrrJ2Ge7opjks5uNESmgaJpZM4VGeIq.

I agree with this issue, but I'm afraid fixing it will break existing contracts for consumers of this API. Can we enumerate the repos of code that use this API, and check the for compatibility? If we can validate it won't break anything downstream, I'll take a look at fixing it.

I actually don't care about it anymore and also am worried about Yap breaking at this point.


Thennable/Promise Implentation

Hey, opening this per suggestion in #339
The returned fromFile() Promise is not spec compliant so it does not resolve if you call then or await on it at a later point.
For example, this code hangs forever on the await, but with a standard Promise implementation it would work:
const dataPromise = csv().fromFile('my-data.csv'))

async function doSomeStuff (name) {
  const data = await dataPromise // This will hang forever!

  return data.find(d => d.name === name)
}

Until a solution is PR'd you can simply do this if you'd like to start parsing but don't want to immediately wait for the result:
const dataPromise = csv().fromFile('my-data.csv')).then((result) => Promise.resolve(result))

async function doSomeStuff (name) {
  const data = await dataPromise // Yay, it works now!

  return data.find(d => d.name === name)
}


tarfile add method bombs because lstat returns a uid of None

When you call tarfile's .add method, it calls lstat underneath the scenes. When using fake_filesystem_unittest, the lstat call returns an st_uid value of None. This causes tarfile to raise an exception.
The following example demonstrates the problem. I tested with version 3.5.
import os
import tarfile
import unittest

from pyfakefs import fake_filesystem_unittest as pyfakefs_unittest

class TestExample(pyfakefs_unittest.TestCase):
    def setUp(self):
        self.setUpPyfakefs()
        self.fakefs_dir = '/fakefs'
        self.fake_src_dir = os.path.join(self.fakefs_dir, 'src_dir')
        self.fake_src_file = os.path.join(self.fake_src_dir, 'src_file.txt')
        os.makedirs(self.fakefs_dir)
        os.makedirs(self.fake_src_dir)
        self.fs.create_file(self.fake_src_file)

    def test_tarfile(self):
        archive_filepath = os.path.join(self.fakefs_dir, 'myarchive.tar.bz2')
        with tarfile.open(archive_filepath, "w|bz2") as tf:
            tf.add(self.fake_src_dir)

if __name__ == '__main__':
    unittest.main()

@ut-adamc - I made the above-mentioned changes. st_uid and st_gid are now set to 1 by default, and you can change the current user/group id.
Please check if that helps with your problem (I can't reproduce this, as I'm currently working under Windows).

Thanks. It does get past that error. There is still another error, which I think is due to tarfile saving the prior value of the open function as "bltn_open" and then using that to open the tarfile.
I think for the tarfile module to work, you would also need to hook bltn_open.

This is the kind of thing that is difficult to hook generically. Using the modules_to_reload argument in setUpPyfakefs may help with that case - this is done automatically with thetempfile module, as it also uses a cached open.

I just looked at the tarfile source code, and the line you have been mentioning is:
from builtins import open as bltn_open

This is not as tricky as I thought - maybe we can handle this generically.
Anyway, setUpPyfakefs(modules_to_reload=[tarfile]) could also work, I'm interested the here if that helps you.

@ut-adamc - I added code to patch this kind of import automatically. Can you please check if the code in master works for you?


chore: fix CI script to work when 'require'ing nested modules

Afraid I'm now going out for the evening, but I can't merge this yet because the CI builds are being slow. Feel free to merge it in my absence!

The new changes seem to get confused with:
lib/gui/modules/analytics.js requires '../../../bower_components/mixpanel/mixpanel-jslib-snippet.js' module which can't be located

Isn't that warning correct, given that bower has now been removed??

Mended.

Isn't that warning correct, given that bower has now been removed??

Not really. Bower is being removed by https://github.com/resin-io/etcher/issues/1208, but that PR didn't get merged yet.

EDIT: Ah, oops. I also 'refactored' the script in this PR to only look for require('.json'), I didn't realise there'd also be require('.js') !?

Not sure what you mean. Is there a require('*.js') or a require('*.json') call somewhere in our code? I don't think CommonJS is able to resolve that correctly. What are we doing in this script in that case?

Not really. Bower is being removed by #1208, but that PR didn't get merged yet.

Yeah, I developed this PR against a local clone of #1208 and forgot to test directly against master, which is how this mistake slipped through (on the first iteration of this PR).

Not sure what you mean. Is there a require('.js') or a require('.json') call somewhere in our code?

Apologies for the confusion. I meant I'd adapted the code to match against \.json$ e.g. require('package.json'), which meant that it broke when it came across e.g. require('../../../bower_components/mixpanel/mixpanel-jslib-snippet.js'). Fixed by updating the regex to \.js(on)?$.


File consistently marked as dirty despite not being so.

A file in my build tree is getting repeatedly marked as dirty.  My first thought is "Why is this even registered as an edge?"
I can't find any instances of Font.h when grepping through the build directory generated by cmake. I assume it's an inferred input since it is included in AssetInterfaces.cc. Interestingly, none of the other files that include Font.h experience the same issue. Could this be a ninja bug, or is there something I should check?

You'll need to provide your entire ninjafile, and probably a lot of additional files, to allow other people to reproduce this problem on their machines.

Reproduction should be possible by building this unless it's a local problem. It only compiles under msvc at the moment. If there's anything I can do to pinpoint the problem I'd be happy to look into it because I have a suspicion that it's local issue.
Here are some relevant ninja files. I looked through them a bit, but nothing sticks out to my untrained eyes.
build.ninja.txt
ninja_deps.txt
ninja_log.txt

I'm sorry for not getting back you. Github never notified me.
I won't make any promises, but I might be able to investigate this in a couple weeks

Since I'm probably going to be pushing to master again and it would be good to have it referenced here. Here's a working commit I remember experiencing the problem on. https://github.com/Underdisc/Varkor/commit/da0bd436cdca5d6f6285bda9f1b288732b798e4d


Define MSRV and enforce on CI

Just currently doesn't build with rust versions prior to 1.52, due to the use of str::rsplit_once in shebang.rs, reported in #960.
We should define a MSRV policy, and enforce it on CI so we don't accidentally break supported Rust versions.
The use of str::rsplit_once isn't crucial, so I'll revert it, and see how much work it is to get Just working with older versions of Rust. Once we get to a version that would require too much work to support, the version after that will be declared the MSRV.
I'm not sure how the MSRV will be increased moving forward. I'll try to be conservative, but if there are shiny new language features that would greatly benefit the codebase, or if an important dependency bumps its MSRV, that will probably mean that Just's MSRV will get bumped.
MSRV bumps will not be reflected in Just's version.

With some not-so-painful changes, I was able to get everything working on 1.46, so that will be the MSRV going forward.

What is MSRV btw?
I found
MSRV | Modified Semi-Solid Rappaport Vassiliadis (microbiology)
MSRV | Multiple Sclerosis-Associated Retrovirus
MSRV | Message Switch Rekeying Variable (US DoD)
MSRV | Multi Sensor Relief Valve

MSRV stands for "minimum supported Rust version", so just the earliest version of the Rust compiler that the project works with.


Set/Unset required field property

can I set the required property for a field based upon an event?
I can't figure out how to do it.

I'm closing this issue since it is probably outdated — Please reopen if you still need help on this. Thanks


property attribute $interval

In the past days, working with the Homie Convention, I on occasions thought, this [...] is missing, on deeper though it was not. But, there is always a but, I now ran into an "omission" that I think would be a useful addition to the convention:
homie / device ID / node ID / property / property attribute
Topic: $interval
Direction: Device --> Controller (Device <--- Controler?)
Description: A string containing the description of the temporal behaviour of a node's property. The interval at which messages are published or (not completely sure here) messages are, expected, to be received in case of a settable property.
Valid Values: A string containing the interval in seconds (float) or the string "on-change", "once" (other strings ...?)
Retained: Yes
Required: Yes (? prefer no)
Property attribute:
$interval is an attribute of a property, not a property of a node (although a node could have). It tells us something about the data being published. My CO2 sensor measures every 5 seconds, I have the data published ever 10 seconds and a min, avg, max every 5 minutes.
Some use cases:
Monitoring;
A client can be written that monitors the timely arrival of data packets and responds to the situation.
Time series;
Even if it would be easy to add time stamps to data it is a bit clumsy to write time series to a database row by row by time stamp. Easier is it to write only a start and or end time and write the data to an array in the DB, that though needs a regular and known interval for the data. A published $interval is this and makes it possible to for a client to note missed packets and write null data in such an array.
A series Timestamps itself will not tell us whether they come in at the set or required interval.
State changes;
During a process the state of a sensor can change and with it its publishing frequency. With a (re-)published interval clients can adapt.
Stale Data;
As all messages are published retained a newly active client can determine by use of the $interval whether the first dataset received by the broker is stale and discard it in that case.

I really like the idea of setting the intervals from the controller to the device, so the controller can ask for more data if it's necessary.
Example:
Normally, you only want to read the temperature of a room every minute or so. But if some sensor detects smoke, you'd like to switch the temperature sensors to a higher frequency, so you can detect where the fire is propagating.
You could do this filtering on the server side, but if it's running over GPRS it's way better to ask more only when needed.
I do something similar to this with broadcasts. If there's a "critical" broadcast, all sensors switch to critical mode, sending as much data as they can, and when things go back to normal I fire a "normal" broadcast.

... and then the next morning I'm measuring the efficiency of my boiler, guess what, the interval is not in seconds but I measure seconds looking at a 1°C interval, being a property of the node.

I don't see the limitation of the (soon-to-be-final) Homie convention. Although there's a recommendation of the units, an "interval" or "time-base" attribute is allowed. You can even make it a settable attribute.
homie/686f6d6965/temperature/$type → temperature
homie/686f6d6965/temperature/$properties → degrees,interval
homie/686f6d6965/temperature/degrees/$settable → false
homie/686f6d6965/temperature/degrees/$unit → C
homie/686f6d6965/temperature/degrees/$datatype → float
homie/686f6d6965/temperature/degrees/$format → -20.0:60
homie/686f6d6965/temperature/degrees → 12.07
homie/686f6d6965/temperature/interval/$name → "interval"
homie/686f6d6965/temperature/interval/$settable → true
homie/686f6d6965/temperature/interval/$unit →  s
homie/686f6d6965/temperature/interval/$datatype → integer
homie/686f6d6965/temperature/interval/$format → 30:300
homie/686f6d6965/temperature/interval → 60

Keep in mind that the homie-esp8266 implementation does not yet adhere to this version of the Homie-convention.

Keep in mind that the homie-esp8266 implementation does not yet adhere to this version of the Homie-convention.

Dat Bert is where I struggled several times now. This morning I did exactly what you wrote above with a bit of "faking it" where "interval/$unit" as a whole becomes a property of temperature instead of "interval" / "$unit"

in the super-car example though one would have some trouble?
homie/super-car/engine/temperature/$interval → "1s"

You have to respect topology:
homie/super-car/engine/temperature/$interval → 1s is actually
homie/super-car/engine/temperature/interval/$name → temp interval
homie/super-car/engine/temperature/interval/unit → s
homie/super-car/engine/temperature/interval → 1

These settings are bound to the attribute (= interval) and not to the property (= temperature, oil pressure) and certainly not to the node (= engine). So you can have different units and different interval value for each property.
My heating boiler could have the following for example:
homie/heating/outside/temperature/interval → 15m
homie/heating/boiler/temperature/interval → 30s
homie/heating/boiler/waterpressure/interval → 24h
homie/heating/hotwatertank/temperature/interval → 1m
homie/heating/hotwatertank/solartemperature/interval → 30s

All have different units, different values and possibly different ranges. The friendly names are always the same as all the different intervals can be identified by it's parent property.

That Bert is clear and obvious from the mqtt point of view. The Homie Convention in that respect though gave me the strong impression that it is limited to one node property without sub properties before the property attribute. Hence my struggle.
homie / device ID / node ID / property / property attribute
versus
homie / device ID / node ID / property [/sub property/...] /  property attribute

hmmm, now I'm confused myself. 😕

Yes, it is a property attribute, regardless of having multiple 'sub'properties or whether they are 'allowed'. This line of thought triggered my initial post.  Then I ran into the problem that it is not just a property attribute with a value, but also a property attribute that needs a unit.

A robot that takes a picture with an $interval of 100m.
You can register the time of a rising temperature in 1°C intervals.
So we end up with $interval & $intervalunit and stuff may accumulate into ugliness. Although $datatype and $format are somewhat similar in this regard.

+1 for me

Just need some more opinions, @ThomDietrich @marvinroger


Fix lint issues.

Pull request summary
Description:
Added:

None

Changed:

None

Removed:

None

Fixed:

None

Testing checklist:

[x] Changes are backward-compatible OR version migration code is included
[x] Codebase compiles with current tsconfig configuration
[x] Tested changes on {choose PUBLIC/PRIVATE} server OR changes are trivial (e.g. typos)

Thanks for doing this! What's your screeps name/slack btw?


Import package from an existing Nuget.Server instance

Is your feature request related to a problem? Please describe.
I'm am trying to do import all packages from our NuGet.Server instance, cause I want to replace it with a BaGet server.
Describe the solution you'd like
It would be great to get an option which downloads all packages from an nuget package server (api v2) to make them available on the BaGet server.
Fallback idea
Provide a way to import nuget package from a folder structure created by the NuGet.Server.
Best regards
Stefan

Hello, here's how you import packages from an existing NuGet.Server instance:

Install nuget.exe
Navigate to the directory containing your NuGet.Server's web.config file using powershell
Run $source = "<NuGet.Server package source>"
Run $destination = "<BaGet package source>"
Run:

if (!(Test-Path "Web.config")) {
  throw "Please run this script in the same directory as NuGet.Server's Web.config file"
}

(& nuget.exe list -AllVersions -Source $source).Split([Environment]::NewLine) | % {
  $id = $_.Split(" ")[0].Trim()
  $version = $_.Split(" ")[1].Trim()

  $path = [IO.Path]::Combine("Packages", $id, $version, "${id}.${version}.nupkg")

  Write-Host "nuget.exe push -Source $destination ""$path"""
  & nuget.exe push -Source $destination $path
}

I'll add this to BaGet's official documentation. Please let me know if you run into any issues!

Hello, thanks for creating the script.
I have extended it a bit and created a .ps1 file. I have added a variable for the path to the nuget.exe, a variable for the apikey you need to push the packages to the BaGet server and a file size limit. The file size limit will be checked before the package will be pushed to prevent an error caused by to large packages.
$source = "<NuGet.Server package source>"
$destination = "<BaGet package source>"
$nugetPath = 'C:\PATH\TO\nuget.exe'
$apiKey = "<BAGET_API_KEY>"
$fileSizeLimit = 10Mb

if (!(Test-Path "Web.config")) {
  throw "Please run this script in the same directory as NuGet.Server's Web.config file"
}

(& $nugetPath list -AllVersions -Source $source).Split([Environment]::NewLine) | % {
  $id = $_.Split(" ")[0].Trim()
  $version = $_.Split(" ")[1].Trim()

  $path = [IO.Path]::Combine("Packages", $id, $version, "${id}.${version}.nupkg")

  if ((Get-Item $path).Length -le $fileSizeLimit) {
      Write-Host "nuget.exe push -Source $($destination) $($path)"
      & $nugetPath push -Source $destination $path -ApiKey $apiKey
   }
   else {
      Write-Warning "Error: file $($path) is bigger than $($fileSiteLimit)"
   }
}


Lighting Panels can't be rotated or dyed

Forge Version
<IP_ADDRESS>59
Galacticraft Version
<IP_ADDRESS>3
Log or Crash Report
No response
Reproduction steps

Place Lighting Panel
Attempt to Rotate
Attempt to Dye

What are you using to rotate/and dye it with?

What are you using to rotate/and dye it with?

Ok, I solved the issue. I thought you could just right click dyes onto the lighting panel.

okay, I'll assume this can be closed as its working as intended. If this is not the case, go ahead and re-open.


Signing / SN

... in order to register the assembly in the GAC.

All future releases will be available as signed and unsigned assembly


Incompatible dataset JSONs files between Mac and Windows

Hi!
This week I've experience a litlle encoding problem in the Dataset JSON files created through the "write" function.
When creating a Mac JSON file, the file can't be opened on a Windows machine. After a few tests, I can say that :

the other way around works (Windows --> Mac)
The file can be read in a text editor
I opened the file created in BBEdit, changed the encoding to Windows and now the file can be opened on the Windows machine.

Specs of my system :

Macbook M1 Max
MacOS 12.3.1
Max 8.3.1

I don't have the specs of the Windows machine, sorry.

Hi @domthibault
I'm now back at my desk. To help me narrow down the problem could you tell me a bit more about the problem you're having? Is it that the files made on the Mac just fail to read properly on Windows (some kind of 'invalid format' error), or that they get read but then you get garbage?

Thanks, that's interesting – we're using the Max SDK to resolve paths, but maybe we're not doing it quite right.
One further question: does the full path to the file in question have any special characters in it?

I will have to check with my student who encountered the problem. Give me a few minutes

@weefuzzy After checking with the student : there were special characters in his path. When he removes them, the file can be loaded in fluid.dataset~.
Thanks for your help!

Thanks @domthibault – glad there's a workaround for now. I'll see if I can get it fixed in the meantime.

Hi @weefuzzy, after further discussing the problem with my student, the problem remains (even without special characters in the path.
I think this info can help you in the search of the bug : when I reencode the Mac file (which is by default Unix LF) to a Windows (CRLF) file, it then works. I'm guessing it has to do with carriage returns and line feeds... :-)

Oh no! So, just so that I have this clear in my mind, if you don't re-encode the file, then your student is getting 'file not found'? But all is well if you do? I'll write some tests and see if I can figure out what's happening...


No intellisense after add Project Reference.

Environment

Steps to Reproduce

Create a solution with 2 Python application projects: ProjectA, ProjectB
Add some python files to each project

From ProjectA, right click Add Reference and select ProjectB.

Verify that ProjectA search paths was automatically updated with a path to ProjectB
Check that in ProjectA you get intellisense against modules in ProjectB

Expected behavior
ProjectA you get intellisense against modules in ProjectB.
Additional context and screenshots
No intellisense against modules in ProjectB

This should be the same issue as https://github.com/microsoft/PTVS/issues/7828


Creating a spell that runs a command

Hi Nathan -
Could you leave a template in the comments for creating a spell that runs a command (i.e. one that does the command /reload (or something along those lines))
Thanks,
Kieran

That kind of stuff is probably a better fit for the wiki.
https://github.com/elBukkit/MagicPlugin/wiki/Command#examples

Thank you :D Looked through the wiki and must of missed that


ERR_CERT_COMMON_NAME_INVALID when accessing deelay.me

It seems that your herokuapp has an incorrect certificate config issue

First up, thanks for this awesome tool! It's the only one I could find that has this kind of functionality.
For a bit of context on the above, I'm using your public service via deelay.me for some demos on this presentation; presentations.danharris.io/web-perf
I get the above error when trying to use your service over https (e.g. https://deelay.me). I'm not sure if https is supported, but would be great if it could be👍
Thanks a bunch!

Thanks for reminding me of this. No, https won't work. I know it's a little bit of a deal breaker, especially with how strict browsers are these days. When I published the website, the browsers didn't care very much about https. It's good to know that people actually want to use it. That should help me with finding motivation and fixing it :)
Some detail about why it is crippled the way it is: Heroku (where the server is hosted) doesn't go well with free certs from let's encrypt. I started setting it up and left half way done when I realized I either need to get a proper certificate or have to change the hosting provider to use Let's Encrypt to make it work.
I'll go back to it and find a different hosting when I have some free time. Any suggestions for an inexpensive cloud to run deelay in a docker container?  I'll keep the issue open and update it when I have some updates.

Ahh, no problems. You can get a free (root) cert through AWS (if you run your docker through Amazon Elastic Container Service).... if AWS works well for you, highly recommend that as a cheap option!

A much faster option is to use cloudflare flexible SSL. It allows visitors to access your site through https on Cloudflare while Cloudflare talks to your server in http
Client  Cloudflare  Your server
For more info:
https://support.cloudflare.com/hc/en-us/articles/200170416-End-to-end-HTTPS-with-Cloudflare-Part-3-SSL-options

Some detail about why it is crippled the way it is: Heroku (where the server is hosted) doesn't go well with free certs from let's encrypt. I started setting it up and left half way done when I realized I either need to get a proper certificate or have to change the hosting provider to use Let's Encrypt to make it work.

FWIW you can use Let's Encrypt on Heroku
https://blog.heroku.com/announcing-automated-certificate-management
heroku certs:auto:enable -a <app name>

Then update your DNS to the new target and you should be good to go.
https://devcenter.heroku.com/articles/automated-certificate-management#migrating-existing-applications

I moved the app to a different hosting provider and added the certs. https is now working.


jwt expire time

Hi ,
Can we refresh expire time on each api request, suppose I set expire time 1 min , and I am hitting authenticated apis every 5 second, So I am not sitting Idle , but after one min token is expired , how can I update expire time on each request. It should be extend at each hit.

No that removes the purpose of a stateless token.  Also you should be throttling your views.
tldr Do not update expire time.
On Sat, Jul 4, 2020 at 4:48 AM Rahul Kumar<EMAIL_ADDRESS>wrote:

Hi ,
Can we refresh expire time on each api request, suppose I set expire time
1 min , and I am hitting authenticated apis every 5 second, So I am not
sitting Idle , but after one min token is expired , how can I update expire
time on each request. It should be extend at each hit.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SimpleJWT/django-rest-framework-simplejwt/issues/264,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOLG4VWLODOQCW6WK6EPPTTRZ3UFZANCNFSM4OQJXOUQ
.


Fix build error of SwiftSample project on Xcode 7.3 (Swift 2.2)

This PR is based on @ishkawa's PR #6346. There were some problems triggering tests on the original PR.

@googlebot , @ishkawa sent his commit as a pull request, so he's OK with it being contributed.