How to stream audio from client to server?

Hi, I am working on a project where I need to stream audio from the user's browser to the remote flask server. On the server, there will be a deep learning model that will return the predictions in real-time. Now, I am really struggling to find a way to stream audio from the user's machine to the server.
I was able to do this locally (without flask) using pyaudio. But, now I am working on a Flask API to achieve the same goal.

I misunderstood the description. After installing the requirements in my anaconda environment. I am getting this error-
Traceback (most recent call last):
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3023, in _dep_map
    return self.__dep_map
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2817, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3014, in _parsed_pkg_info
    return self._pkg_info
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2817, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/v-mohit.saini/anaconda3/envs/flask/bin/flask", line 8, in <module>
    sys.exit(main())
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/flask/cli.py", line 513, in main
    cli.main(args=args, prog_name=name)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/flask/cli.py", line 380, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/flask/cli.py", line 336, in get_command
    self._load_plugin_commands()
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/flask/cli.py", line 332, in _load_plugin_commands
    self.add_command(ep.load(), ep.name)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2444, in load
    self.require(*args, **kwargs)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2467, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 795, in resolve
    new_requirements = dist.requires(req.extras)[::-1]
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2738, in requires
    dm = self._dep_map
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3025, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3034, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3016, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1421, in get_metadata
    value = self._get(path)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1617, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/MarkupSafe-1.1.1.dist-info/METADATA'

Found the problem. Installing the new libraries in previous environment messed it up(I have to set it up again :'( ). Creating new environment is worked. Closing.


feat: granular control over plugin load order and user request

Is your PR related to a specific issue/feature? Please describe and mention issues.
Closes #162 by adding the following features:

Use the sanctum.appendPlugin config key to enable the append operation instead of the default prepend for the plugin.
Disable initial user request by changing sanctum.client.initialRequest to false. A separate method now is exposed in useSanctumAuth composable - init, which will be executed only once regardless of the selected SSR/CSR environment.

Additional context
Since the plugin's dependsOn cannot be configured externally (due to module initialization scope), module options cannot be used either.
Now when we need to ensure that the sanctum client makes the initial user request only after some other plugins are loaded (e.g. i18n), we should define a custom local plugin with dependsOn: ['i18n', 'nuxt-auth-sanctum'] and call init method there manually, for instance:
// my-custom-auth.ts
export default defineNuxtPlugin({
  name: 'custom-auth',
  dependsOn: ['i18n', 'nuxt-auth-sanctum'],
  async setup() {
    const { init } = useSanctumAuth()
    await init()
  }
})

Added a new section to the docs - https://manchenkoff.gitbook.io/nuxt-auth-sanctum/usage/plugin-dependencies. Also updated page with the config definition.


4.x Tracing does not add documented tags requested.type and response.type to spans

Environment Details

Helidon Version: 4.0.1
Helidon SE or Helidon MP
JDK version:
OS:
Docker version (if applicable):

Problem Description
The tracing documentation states that Helidon adds the tags requested.type to the content-read span and response.type to the content-write span.
Yet the code does neither.
Steps to reproduce

Create tracing-app.settings containing:tracing=true
app-type=custom

Run helidon init --version 4.0.1 --props-file tracing-app.settings --batch
cd custom-se
mvn clean package
Start Jaeger. (One way: download it from here https://www.jaegertracing.io/download/, expand it into a known location, then run ./known-location/jaeger-all-in-one
java -jar target/custom-se.jar
curl http://localhost:8080/simple-greet
From a browser:

Access the Jaeger UI: http://localhost:16686
In the Service drop-down, select helidon-tracing-service.
Click the Find Traces button near the bottom left of the screen.
Click on one of the spans labeled "helidon-tracing-service: HTTP Request` which opens up some details
Click on the content-write span. The new details include tags.

The tags list does not include response.type as our documentation claims it should.

curl -X PUT -H "Content-Type: application/json" -d '{"greeting" : "Hola"}' http://localhost:8080/greet/greeting
In the Jaeger UI:
Click the left arrow to return to the trace list.
Click Find Traces again; you should see a new trace for the PUT.
Click on the most recent trace.
Click on the content-read span.

The tags list does not include requested.type as our doc describes.

I have not found references to these tags in any of:

OpenTelemetry tracing
MicroProfile tracing
OpenTracing

There is code in Helidon 3.x that sets the requested.type tag (in webserver/webserver Request.java) and sets response.type (in webserver/webserver Response.java).
The code in 4.x is quite different and adding support for these tags in 4.x would look quite different from the 3.x code.

Tomas reports in private communication that the removal of the tags was intentional, so I have changed this to a docs issue.


update golangci-lint and pin

Install a specific version as suggested on their README so we don't get failures on updates to the repo (see the last two commits to master).

Thanks for fixing this! I've tried that also but ran into problems running this on developer machines.
The build failures on master seem to come from a new check that you also fix in this PR. Maybe we just add that fix here?
Strangely enough, the new build failure now is ginkgo: Command not found...

Had accidentally removed the line where we had installed ginkgo. Added that back


oauth2

Current dependencies on/for this PR:

main

PR #21   👈

This stack of pull requests is managed by Graphite.


segmentation fault error when use jsonpickle to serialize and de-serialize an object

Hi,
I tried to use jsonpickle to serialize an object. But after de-serialization, I met an error: segmentation fault.
Following is my code:
import jsonpickle
import pandas as pd

class MyClass:
    def __init__(self, df):
        self.df = df

df = pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
my_class = MyClass(df)

json_string = jsonpickle.encode(my_class)

f = open('./test.json', 'w')
f.write(json_string)
f.close()

read_path = 'test.json'
f = open(read_path)
json_str = f.read()
obj = jsonpickle.decode(json_str)
print(obj.df)

And I found that if the data type of attribute in the class is list or int, instead of pandas dataframe, jsonpickle works. My python version is 3.8.8.
Is there any way to solve this problem?
Thanks!

Hey there, sorry for the late response (midterms have been difficult lately). I believe you need to register the jsonpickle pandas handlers, there should be instructions on how to do that in the online documentation for jsonpickle. If that doesn't work, please let me know!

Never mind, registering pandas handlers doesn't work. I'll try and figure this out, but it may be difficult.

Never mind, registering pandas handlers doesn't work. I'll try and figure this out, but it may be difficult.

Hi,
I have run the code mentioned in the issue description with adding the pandas handlers and it worked fine for me.
Can you please provide more insights regarding this issue?
here's the code that I ran -> https://colab.research.google.com/drive/1il1cS8S-fFmT_bSyR0_buDfwrxxK6jvf?usp=sharing
Thanks

Ah, the code I used did the following:
import jsonpickle
import pandas as pd

jsonpickle.handlers.register('pandas')

instead of, the following, which works:
import jsonpickle.ext.pandas as jsonpickle_pandas
jsonpickle_pandas.register_handlers()
import jsonpickle
import pandas as pd

Thanks for figuring this out @rishavganguly007, and thanks for testing it @helloyuepang! I'll try and figure out why the first method didn't work, and hopefully I can fix it for 3.0.2.


[ko] Update outdated files in dev-1.26-ko.1 (M34-M41)

Signed-off-by: jongwooo<EMAIL_ADDRESS>Description
Related Issue: #38458
Updated outdated files in content/en/docs/concepts/overview/working-with-objects/.
8 files to be modified

[x] M34.  content/en/docs/concepts/overview/working-with-objects/annotations.md | 1(+XS) 1(-)
[x] M35.  content/en/docs/concepts/overview/working-with-objects/common-labels.md | 1(+XS) 0(-)
[x] M36.  content/en/docs/concepts/overview/working-with-objects/field-selectors.md | 2(+XS) 1(-)
[x] M37.  content/en/docs/concepts/overview/working-with-objects/finalizers.md | 1(+XS) 1(-)
[x] M38.  content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md | 11(+S) 6(-)
[x] M39.  content/en/docs/concepts/overview/working-with-objects/names.md | 2(+XS) 2(-)
[x] M40.  content/en/docs/concepts/overview/working-with-objects/namespaces.md | 24(+S) 13(-)
[x] M41.  content/en/docs/concepts/overview/working-with-objects/object-management.md | 1(+XS) 1(-)

/assign

감사합니다!!
/approve


LOS has TypeErrors when reloaded while there are speakers

The error is ERROR TypeError: Cannot read properties of null (reading 'user')  at get userId

Only happens if the setting for showing the first speaker hint is enabled


mat-select stranger behavior after updating to version 16.2.6

Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description

Reproduction
StackBlitz link:
Steps to reproduce:
1.
2.
Expected Behavior
I'm not sure what happened after upgrading to version 16.2.6
Actual Behavior

Environment

Angular:
CDK/Material:
Browser(s):
Operating System (e.g. Windows, macOS, Ubuntu):

@gurelsoycaner Could you provide a stackblitz reproduction of the issue? It is infeasible to debug this without one. My best guess going just by the images you shared is that the z-index of one of the other elements is greater than the z-index of the mat-select dropdown

It looks like the theme for the select isn't included. See the theming guide for more information https://material.angular.io/guide/theming


Support module search

This supports a very basic search using /modules.json.

Project seems dead. I'll close this.


error during processing Error: Arguments must all be strings

Hmm... After installing this extension and installing nodejs/NPM I keep getting "error during processing Error: Arguments must all be strings". According to Panic software support, it has to be a configuration mistake I made. Any adea what causes this error?

Hi @reiniertc, thanks for writing in with this issue. I haven't updated the extension in a while, as I stopped using Nova for the time being, until it gets a little more robust.
Could you paste a screenshot of your settings here? I'll test this out and see what I need to update to get this fixed ASAP.
Thanks again!

thanks!
here is the screenshot:
https://www.dropbox.com/s/3w4mizbx21th8sj/Schermafbeelding 2021-12-01 om 16.28.11.png?dl=0

I am having the same issue with this plugin. Curios if you have had a chance to look at it.


Create polygons for sites table in BETYdb

Description
The spring planting plots are described in this spreadsheet on google drive

the plot layout is described in the 'experimental plot plan' table, the important bits are:

dimension
value

# rows
32

# rows / plot
2

# plots (2 rows ea)
864

# ranges
54

# columns
16

row width (m)
0.762

plot length (m)
4

row length (m)
3.5

alley length (m)
0.5

I began working with these data in R. The following code uses the above dimensions to generate the points and then convert them to polygons in WKT that would be suitable for entering into the database
(20160421_plots.csv.txt)
except that we need to

[ ] put them in gantry coordinate system and
[ ] convert them to a geographic coordinate system.
[ ] insert them into the database

ultimately we need something like the follw
The integer values for 'plot' are 1-864 are associated with plots in the terraref instance of BETYdb named "MAC Field Scanner Field Plot i"

Here are the sites in TERRA-REF BETYdb that we are trying to update: https://terraref.ncsa.illinois.edu/bety/sites.json
Here are the sql statements in a system that puts (0,0) at the bottom right of the field:
insert_site_geometries.sql.txt
# code in https://github.com/terraref/reference-data/blob/master/scripts/maricopa_field_metadata.Rmd
require(udunits2)
library(dplyr)
library(tidyr)
library(ggplot2)

x_origin = 0
y_origin = 0
length = 3.5
alley = 0.5
row_spacing = 30
ranges = 54
columns = 16

row_spacing_cm <- ud.convert(row_spacing, 'in', 'm')

dx <- 2 * row_spacing_cm
dy <- length

x <- x_origin + (1:columns - 1) * dx
y <- y_origin + (1:ranges - 1) * (length + alley)

grid <- data.frame(expand.grid(x,y)) %>%
  transmute(plot = 1:(ranges*columns), x1 = Var1, y1 = Var2, x2 = x1 + dx, y2 = y1 + dy) #%>%
  #tidyr::gather(key = 'lat', )

theme_set(theme_bw())
ggplot(data = grid) +
  geom_text(aes(x1+dx/2, y1+dy/2, label = plot), size = 2) +
  geom_point(aes(x1,y1), size = 1, alpha = 0.4) +
  geom_point(aes(x2,y2), col = 'red', size = 1, alpha = 0.4) +
  ggtitle('gantry plot layout') + xlab('x') + ylab('y')

grid2 <- grid %>% gather(key, value = x, starts_with("x")) %>% gather(key2, value = y, starts_with("y"))   %>% select(-starts_with("key"))  %>% arrange(plot) %>% setDT

library(spatstat)
library(wellknown)
write.csv(merge(grid, plots, by = 'plot'), file = '20160421_plots.csv.txt')

insert <- plots[,list(sql = paste0('update sites set geometry = ', geometry, 'updated_at = now(), where name = ', paste0("'MAC Field Scanner Field Plot ", plot, "';")))]

writeLines(insert$sql, con = file('insert_site_geometries.sql.txt'))

Related issues:

https://github.com/terraref/reference-data/issues/32
https://github.com/terraref/computing-pipeline/issues/109
https://github.com/terraref/reference-data/issues/51

@JeffWhiteAZ or @smarshall-bmr

could you please provide information for mapping these plots to the gantry (0,0) coordinates?
do you have similar dimensions for the second planting?

https://docs.google.com/spreadsheets/d/1eQSeVMPfrWS9Li4XlJf3qs2F8txmddbwZhjOfMGAvt8/edit#gid=883764630
https://docs.google.com/spreadsheets/d/1eQSeVMPfrWS9Li4XlJf3qs2F8txmddbwZhjOfMGAvt8/edit#gid=1433617856
These are the plot locations as given in the gantry coordinate system for the first and second planting respectively. The y coordinates change a negligible amount as to the point that the y coordinates are given as the same between plantings. The x axis is offset to the north approximately 50cm for the second planting.

@dlebauer I have a few questions related the R code:
1 Q1: the plot number in the picture given by ggplot2 in the R code does not match with the excel below (is it important?)
https://docs.google.com/spreadsheets/d/1QQaWc0UaQQKfEtnSO1G2za8tKU2huC0_VYMBqm5CKAo/edit
For example, the location of plot 17 is not consistent with the excel.
2 Q2: what is the correct lat lon for the four corners that we should build the projection on? I have design a projection system using the lat lon of the corners provided before.
Location	Latitude	Longitude
S-E	33.07459197	-111.97480975
S-W	33.07459172	-111.97504692
C-W	33.07547558	-111.97504675
C-E	33.07547550	-111.97481017
N-W	33.07642747	-111.97504728
N-E	33.07642819	-111.97481064
However, I found that the corners points I used are actually the six points SW SE CW CE NW NE  in the sheet below. I am not sure what is the relationship between the three lat&lons.
https://docs.google.com/spreadsheets/d/1pTgmpf9kltYW1SHo_K51cynVTYs6-JdS6rt4usfwV10/edit
3 Q3: why the bounding box is much larger. The bounding box I got from the grid calculated in the R code is much larger than the bounding box of the corner points in the google sheet mentioned in Q2.

boundingbox=c(min(grid$x1),min(grid$y1),max(grid$x2),max(grid$y2))
boundingbox
[1]   0.000   0.000  24.384 215.500

4 Q4: I saw many people talking about gantry coordinates and lemnatec coordinates. I wonder what's the relationship between these different coordinates. Also, how they are related to coordinates with SW corner as (0,0) .
5 There is an example of how to add lat1 and lon1 for X1 and Y1, add lat2 and lon2 for X2 and Y2 to grid data frame using the projection I provided before.
require(proj4)
options(digits = 15)
x1y1=cbind(grid$x1,grid$y1)
x2y2=cbind(grid$x2,grid$y2)
proj= '+proj=tmerc +lat_0=0 +lon_0=-111.843479 +k=1 +x_0=12285.092664 +y_0=-3661028.344732 +datum=WGS84 +units=m +no_defs '
latlon1=project(x1y1, proj, inverse = T, degrees = TRUE, silent = FALSE, ellps.default="sphere")
latlon2=project(x2y2, proj, inverse = T, degrees = TRUE, silent = FALSE, ellps.default="sphere")
grid$lon1=latlon1[,1]
grid$lat1=latlon1[,2]
grid$lon2=latlon2[,1]
grid$lat2=latlon2[,2]

But I failed to run the code from grid2 <- grid %>% gather(key, value = x, starts_with("x")) %>% gather(key2, value = y, starts_with("y"))   %>% select(-starts_with("key"))  %>% arrange(plot) %>% setDT . I am not sure what that part means.

@tingli3 I answered your questions inline above

@dlebauer Thank you.
The true SW and SE lat & lon will be very important in the projection definition. If USDA is the truth lat & lon for the SW and SE, I need to update the projection based on it and make sure after projection the SW will be (0,0) and SE will be something like (2*.***, 0). Please tell me whether my understanding is correct.
However, here the coordinate system we generated using R code start at (0, 0) and ends at (24.384, 215.500). The four points given by USDA is approximately 27.5m * 110m. If that is the case, I wonder whether the four corners of the gantry plot layout should match with the 4 points of USDA. Specifically, whether the SW point of the plot1 is the SW point of USDA points and SE point of the plot16 is the SE point of USDA points, and how to handle the difference.

@tingli3 there are two different issues.
First, is to determine the coordinate reference systems. That should be covered in issue https://github.com/terraref/reference-data/issues/32. There are four (or more) points used to align the two (MAC and USDA) GPS systems. These are in concrete posts around the edge of the field. There are also points used to align the gantry coordinate system with the MAC RTK GPS.
Second is this issue, to determine the bounding boxes of the individual plots in these different coordinate reference systems.
I will send you an email with a complete discussion of how these points were collected.

@dlebauer So, what is the X and Y in the R code corresponds to ? Where is the true (0,0) ?

@tingli3 First, I apologize on behalf of all involved that this is such a mess.
But to try and clarify, there are a three (X,Y) coordinate systems

the gantry / lemnatec system that has reversed X. These are
the 'corrected' X,Y with (0,0) on the SW reference point. This does not yet exist, but is covered in (https://github.com/terraref/computing-pipeline/issues/109) that this would be helpful for plotting and because humans are used to X increasing to the right / East
the values used in the R code and experimental design. These are in relation to the corner of the planted area, which will change each season.

To map the plots (3) to the gantry (1), it is necessary to align the X,Y pairs in the R code with the X,Y measured coordinates provided by @smarshall-bmr in this spreadsheet: https://docs.google.com/spreadsheets/d/1eQSeVMPfrWS9Li4XlJf3qs2F8txmddbwZhjOfMGAvt8/edit#gid=883764630

@dlebauer
So, the first step is to convert the gantry system based on its bounding box to make sure that X starts at 0 and increases to the East using something like x'=-y+22.135 and y'=x-3.8. But this assumes that the six point given is the bounding box (22.135m*203.5m) of the gantry field.
If this is not the case, I think the first step is to find out the bounding box of gantry field , and how the gantry should be converted (set 0,0 X-axis Y-axis for it), and how it is relate the to the XY of the plots. Is it possible to use the min and max in the https://docs.google.com/spreadsheets/d/1eQSeVMPfrWS9Li4XlJf3qs2F8txmddbwZhjOfMGAvt8/edit#gid=883764630 to get bounding box of the gantry coordinates? If this is the case, which sheet (RangeCentersSorghum1 or RangeCentersSorghum2) should be used? And it seems that range 1 and range 54 does not have value in RangeCentersSorghum1 and the value in this two sheet are different.

@tingli3 I think it would help if you could create a map with the different coordinates reference systems and available reference points. I am confused that x'=f(y) instead of f(x), and it would really help to clarify. Otherwise, I am afraid that the discussion we are having here will add to the confusion.
... and as you sort this out please document the issues and solutions https://github.com/terraref/documentation/issues/7

@tingli3  RangeCentersSorghum1 and RangeCentersSorghum2 are for the first and second planting seasons, respectively.
I believe the GPS reference points that compare UA MAC and USDA GPS's correspond to concrete pylons that are in the ground a few meters outside of the planted area.

@dlebauer
Using x'=f(y) instead of f(x) is to transpose X and Y as the X goes S to N in the gantry coordinates (the X and Y of the gantry should be inversed).
For the two planting seasons, are the coordinates recorded in the same gantry coordinate system? Which one should be used? Finding the exact bounding box of the gantry data is very important.

@tingli3
For the two planting seasons, the coordinates are recorded in the same system. We will need to recreate the plot boundaries each time a new crop is planted, and the bounding box of the planted area as well as the plots themselves will change.

@dlebauer
From the information, I assume we are going to use one system and put all the data with different bounding box into it.  Also, the two planting seasons at least must have to have the same (0,0) point (though we do not know where it is  ) and XY axes (I think this should be true). That means the same gantry coordinate in both two seasons should exactly point to the same point. Is this correct?
And, as the bounding box is changing and (0,0) point is unknown, the new gantry coordinates with X increasing to the east will also not start exactly at (0,0). The (0,0) point of the plots may corresponding to a gantry coordinate not exactly (0,0). Is that OK?

Yes that is okay. Only the plots change. I think we should discuss in
person, but please map what you have so it is easier to understand.
On Fri, Sep 30, 2016 at 5:51 PM Ting Li<EMAIL_ADDRESS>wrote:

@dlebauer https://github.com/dlebauer
From the information, I assume we are going to use one system and put all
the data with different bounding box into it. Also, the two planting
seasons at least must have to have the same (0,0) point (though we do not
know where it is ) and XY axes (I think this should be true). That means
the same gantry coordinate in both two seasons should exactly point to the
same point. Is this correct?
And, as the bounding box is changing and (0,0) point is unknown, the new
gantry coordinates with X increasing to the east will also not start
exactly at (0,0). The (0,0) point of the plots may corresponding to a
gantry coordinate not exactly (0,0). Is that OK?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/terraref/computing-pipeline/issues/174#issuecomment-250870112,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAcX5zUnkX-4CWN05Er4XA2Z_G9y9ozLks5qvZJpgaJpZM4KFRZU
.

I was very confused and when you (@tingli3) pointed out x'=f(y) I realized that I misunderstood the gantry coordinate system.
But drawing a map helped me. This may be incorrect, but it is my current working hypothesis of how the different coordinate systems overlay.
If x goes positive to the north and y goes positive to the east, we don't need a coordinate system with x going east. That was only because if x increases to the west / to the left, images, plots, and perspective are inverted.
So, we should review these maps and be able to encode this in a database and use it to subset data and visualize the experimental design.

[p.s. Did anyone else ( @pless @LTBen @rjstrand @smarshall-bmr @doctorroboto et al ) catch on that x is increasing to the north?]

This R code corrects the order of plotID and add column and range ID to each plot. Grid[i,] corresponds to plot i. The X1, X2, Y1 and Y2 need to be updated using the true gantry coordinates.
require(ggplot2)
require(udunits2)
x_origin = 0
y_origin = 0
length = 3.5
alley = 0.5
row_spacing = 30
ranges = 54
columns = 16
row_spacing_cm <- ud.convert(row_spacing, 'in', 'm')
dx <- 2 * row_spacing_cm
dy <- length
x <- x_origin + (1:columns - 1) * dx
y <- y_origin + (1:ranges - 1) * (length + alley)

grid <- data.frame(expand.grid(x,y))
names(grid)=c('x1','y1')
plot = 1:(ranges*columns)
grid=cbind(plot,grid)
grid$x2=grid$x1+dx
grid$y2=grid$y1+dy

grid$column=(grid$plot-1)%%16+1
grid$range=floor((grid$plot-1)/16)+1

for ( i in (1:27)*2 )
{
  grid[((i-1)*16+1):(i*16), 1]=c((i*16):((i-1)*16+1))
}

grid=grid[order(grid$plot),]

theme_set(theme_bw())
ggplot(data = grid) +
  geom_text(aes(x1+dx/2, y1+dy/2, label = plot), size = 3.5) +
  geom_point(aes(x1,y1), size = 1, alpha = 0.4) +
  geom_point(aes(x2,y2), col = 'red', size = 1, alpha = 0.4) +
  ggtitle('gantry plot layout') + xlab('x') + ylab('y')

This is the R code for projection. Lat&lon can be returned through gantry2latlon(gantry_x, gantry_y) e.g. gantry2latlon(3.8,0). How the choose of the point (currently CE) and projection will effect the error needs to be experimented later.
require('proj4')
options(digits=15)
gantry2latlon <- function(gx,gy)
{
  #you can replace CE with other points such as SW SE ...
  CE=c(-111.974810166667,33.07547550)
  CE_gantry=c(101.750,0)
  dx_g=gx-CE_gantry[1]
  dy_g=gy-CE_gantry[2]
  proj= '+proj=tmerc +lat_0=0 +lon_0=-111.843479 +k=1 +x_0=12285.092664 +y_0=-3661028.344732 +datum=WGS84 +units=m +no_defs '
  #or you can use
  #proj= '+proj=tmerc +lat_0=0 +lon_0=-111.974928458333 +k=1 +x_0=50000 +y_0=0 +datum=WGS84 +units=m +no_defs '
  #proj='+proj=utm +zone=12 +ellps=GRS80 +datum=NAD83 +units=m +no_defs'

  CE_UTM=project(CE,proj)
  x_UTM=CE_UTM[1]-dy_g
  y_UTM=CE_UTM[2]+dx_g
  #return (round(project(c(x_UTM,y_UTM),proj,inverse=T),15))
  return (project(c(x_UTM,y_UTM),proj,inverse=T))
}

Next:

generate polygons to be inserted to betydb in postgis sql statements. send to Scott.
two site tables for two seasons. tables use USDA GPS reference points.
https://docs.google.com/spreadsheets/d/1eQSeVMPfrWS9Li4XlJf3qs2F8txmddbwZhjOfMGAvt8/edit#gid=883764630

https://docs.google.com/spreadsheets/d/1eQSeVMPfrWS9Li4XlJf3qs2F8txmddbwZhjOfMGAvt8/edit#gid=1433617856

work out sql query statements on querying the tables to output MAC and gantry coordinates.

Due Oct 14th.

From Andy French on Sept 2, 2016:

Use the USDA RTK system as the base reference, it is likely more accurate than the M&S (this is not CASE IH as understand). The USDA RTK system was checked a while ago (2007 so not so recently) three AZ base stations  (designations AZMO, AZSC, AZCLL) and differed from the NAD83 reference frame by 2, 1.2, 1.9 cm for the X,Y, Z coordinates respectively. We routinely check our locations against the USGS National Map and locations are typically within ~2cm

  We (Bill Luckett and I) checked the location of the M&S base station with the USDA rover (with some error because the M&S antenna is behind a locked fence to the west of the scanalyzer field). Thanks to Pedro for additional information we’ve confirmed the 2.5 m discrepancy is due to an inaccurate position used in commissioning the M&S base. This discrepancy appears to be constant across MAC fields. I suspect the M&S system precision is excellent even though its accuracy is not. Because of these factors, moving between the ‘best’ coordinates and the M&S coordinates is a matter of book-keeping to make sure one records your GPS source GPS.

Bill and Bob Strand have collected locations of the concrete piers using the USDA RTK system. Thus a linear transformation matrix can be easily created to move between scanalyzer coordinates and the currently best possible UTM13 coordinates. This matrix and the offset matrix from #2 can be stored in your PostGIS system.

  At some unknown time in the future the inaccuracy of the M&S system will have to be addressed, but as this is not under UA control, cannot be forced. To me this represents a risk, maybe a small one, that one day we will be surprised to find the bias go away. My opinion is that it would be better to maintain more control over your geolocation positioning accuracy.

@tingli3 Upon Andy comment 3: need to confirm with him the EPSG number:
http://spatialreference.org/ref/epsg/26948/
http://spatialreference.org/ref/epsg/26949/
http://spatialreference.org/ref/epsg/26950/

@tingli3 Need to clarify if the 2.5m is on x or y or both directions by going through the email. Ask if needed.

@tingli3 Upon Andy comment 2: need to ask Andy or Bob whether the Lematec coord derived using the linear regression results for MAC coord system has been corrected in the definition of b0, b1 and b2.

@tingli3 Need to confirm with Andy and Bob or @dlebauer whether the lat lon info associated with Lemnatec gantry raw data is the corrected values based on Andy's comment 1.

@yankiu-chn are you are asking if the gantry points were corrected to the USDA coordinates? I believe the answer is 'no', as they are labeled UA MAC but maybe @rjstrand @JeffWhiteAZ or @Andrade-Pedro can comment

@dlebauer I meant column B and C from row 4-9 in UA-MAC-F13_RTKRef_9-2-2016 sheet.

@dlebauer ok, so UA-MAC-F13_RTKRef_9-2-2016 is all about MAC coord. I revised my comment.

@yanliu-chn yes, it is from the UA-MAC RTK GPS; the comments say 'from the tractor', which implies the UA-MAC data.
@JeffWhiteAZ will we be getting any data (other than for correction of the UA-MAC system) from the USDA RTK GPS coordinates?

Here is another example of Rcode using the linear regression method to get the lat and lon.
All the files need are in projection.zip.
data=read.csv("coords.csv")
require('proj4')
options(digits=15)
gantry2latlon_lm <- function(gx,gy,data)
{

  proj= '+proj=tmerc +lat_0=0 +lon_0=-111.974928458333 +k=1 +x_0=50000 +y_0=0 +datum=WGS84 +units=m +no_defs '

  lonlat=cbind(data$Longitude,data$Latitude)
  projectxy=project(lonlat,proj)
  data$projectX=projectxy[,1]
  data$projectY=projectxy[,2]

  fitX=lm(projectX~gantryX+gantryY,data)
  fitY=lm(projectY~gantryX+gantryY,data)

  predictdata=data.frame(cbind(gx,gy))
  names(predictdata)=c('gantryX','gantryY')
  X=predict(fitX,predictdata)
  Y=predict(fitY,predictdata)

  result=project(cbind(X,Y),proj,inverse=T)
  return(result)
}

Short term, we should not receive more data using the USDA GIS system. In the medium term, possible sources of such data would be:

  Mapping additional concrete pylons, etc. under the Gates Foundation project.

  Specific measurements made by A French or others using USDA equipment

Note also that any work done with WAAS GPS (e.g. in UAVs) should match the USDA coordinates, albeit with lower accuracy (0.25 to 2 m RMSE).

Jeff

From: David LeBauer<EMAIL_ADDRESS>Sent: Wednesday, October 5, 2016 1:50 PM
To: terraref/computing-pipeline<EMAIL_ADDRESS>Cc: White, Jeffrey - ARS<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [terraref/computing-pipeline] Create polygons for sites table in BETYdb (#174)
@yanliu-chnhttps://github.com/yanliu-chn yes, it is from the UA-MAC RTK GPS; the comments say 'from the tractor', which implies the UA-MAC data.
@JeffWhiteAZhttps://github.com/JeffWhiteAZ will we be getting any data (other than for correction of the UA-MAC system) from the USDA RTK GPS coordinates?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/terraref/computing-pipeline/issues/174#issuecomment-251795404, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AD0shjfhuiGdaDZWABitaM2X80CJirzgks5qxA2JgaJpZM4KFRZU.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

@tingli3 @dlebauer Summary of today's working meeting:
https://github.com/terraref/documentation/blob/version1/spatial-reference-system-data.md
@tingli3 please read the sites table section to be clear about the input and output.

@yanliu-chn
Why UTM 13 zone?
UTM 13 zone is from -108 degree to -102 degree.
I think UTM 12 zone is more suitable as the longitude is around -111degree.

Not sure of context of question, but AZ is mostly within UTM 12.

Jeff

From: Ting Li<EMAIL_ADDRESS>Sent: Thursday, October 6, 2016 2:09 PM
To: terraref/computing-pipeline<EMAIL_ADDRESS>Cc: White, Jeffrey - ARS<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [terraref/computing-pipeline] Create polygons for sites table in BETYdb (#174)
@yanliu-chnhttps://github.com/yanliu-chn
Why UTM 13 zone?
UTM 13 zone is from -108 degree to -102 degree.
I think UTM 12 zone is more suitable as the longitude is around -111degree.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/terraref/computing-pipeline/issues/174#issuecomment-252089279, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AD0shgfUIR-8mT2hAXWS64aHN1Bqvwx6ks5qxWNngaJpZM4KFRZU.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

I mean in this document https://github.com/terraref/documentation/blob/version1/spatial-reference-system-data.md. All projection is in UTM 13

Is a typo, it should be UTM Zone12

I inherited the type from Andy's earlier email. :) I have corrected it in both the document MD and the comment in this issue.

sql.zip
This is an example of generating SQL statements with range.csv and row.csv as example input. Please make sure that range.csv (must have 54 entries) and row.csv (must have 32 entries) have correct values (For the row.csv, I assume all the row start at y=0 and each plot covers one column = two rows). I am using the values of planting season 2 in these two files but I do not quite understand the 2016_F13_TERRA_sorghum_field_book.xlsx. The values in the two files should be corrected by someone.
In the SQL part,
plotid is the ID of each plot; col is the column ID; range is the range ID; geomgantry is a polygon of the plot directly using (X,Y) gantry coordinates (How to handle the inversed X and Y in gantry?); X and Y is the gantry coordinate of the SW point of each plot; gantryw is the length of the shorter edge and gantryl is the length of the longer edge in gantry coordinates; geousda is a polygon of the plot using lat and lon generated from https://github.com/terraref/documentation/blob/version1/spatial-reference-system-data.md; lon and lat is the longitude and latitude of the SW point of each plot; utmw is the length of the shorter edge and utml is the length of longer edge calculated by project the lat&lon into UTM.

@gsrohde could you please review and run the SQL boundaries?

@tingli3 thank you for doing this.
Could you please

fix the insert statements so that they work on BETYdb, e.g. if you look at the original post / examples the insert statement should be something like

paste0('update sites set geometry = ', geometry, 'updated_at = now(), where name = ',
    paste0("'MAC Field Scanner Field Plot ", plot, "season ", season, " ';"))

run this for both the first and second planting seasons

I've added the border information for season 1 in the spreadsheet

Modify the script so that it will work for any number of rows and columns

@tingli3 also, could you please check your code and inputs into the repository?

@tingli3 please also add elevation 353

@gsrohde could you test the sql statements and let us know what to improve?

@dlebauer could you ask the right person to define the format in the excel so that we know for sure how to extract columns in order to figure out the number of rows and columns explicitly and implicitly? An explicit way is to specify the number of rows and columns in the sheet and tell us where the numbers are.
And, if you want to derive the plot coord for any number of rows and columns, tell us the formula to derive the plot id from the range and column id.

@yanliu-chn will the ranges and rows worksheets that I created work? If not, please go ahead and define what you need, and we can work backward from there.
@JeffWhiteAZ is there a standard way of naming / numbering plots based on the ranges and columns? Is there a reason that the numbers increase alternatively going L-->R and then R--> Left going North?

@dlebauer i think we need one single format that does not change across years. In that format, we should have enough info to derive the geom and plotid. That's all.
Our current assumption on the range-column sheet is:

the range column defines range index, starting from 1, incremented by 1. The largest number we can get, say X, is used to calc the number of ranges: X-1+1;
the same assumption for the column column
There is no gap along column direction
The deference b/w any consecutive columns is the column size
range must have the x coord for the south and north to figure out the range size

If this assumption holds for all the sheets to be produced in the future, we are fine. We don't want to expect variations such as for some year, the column Y value becomes the column size, not gantry Y; column patterned as 1 3 5 9 11 15 and the index increased by 2.

@yanliu-chn please move forward with this assumption - since it doesn't currently exist, once you specify the format (in documentation) then it becomes the de-facto standard!

I tweaked the code to insert polygons for season 2 by adding
  betydb_insert[[i]] <- paste("INSERT INTO sites (sitename, geometry, updated_at, created_at) VALUES (",
                         paste0("'MAC Field Scanner Field Plot ", grid$plot[i], " Season 2', "),
                         geom_latlon, ", now(), now());")
}

to the final for loop.
and then filled out the information by running
update sites set city = 'Maricopa', state = 'Arizona', country = 'USA', time_zone = 'America/Phoenix', greenhouse = 'f', user_id =<PHONE_NUMBER>, updated_at = now() where sitename like 'MAC Field%';

the geom_gantry information will be useful for extractors that use the gantry coordinate system, but it does not fit into BETYdb. Where can it be stored? Or can we modify the geometry column so that it can optionally return these points?

@dlebauer (and @tingli3) Please don't include the created_at and/or updated_at columns in the insert and update statements!  A trigger function handles these automatically so that created_at and updated_at are set to the current time when a row is inserted, and updated_at is reset to the current time when a row is updated.  Setting them explicitly overrides this behavior, and using now() sets the column to machine time, not UTC time.  (There is a function I defined, utc_now(), which will return UTC time, but you don't need to use it in this case.)

@yanliu-chn I think the lines in the MAC --> USDA (M2U) section of https://github.com/terraref/documentation/blob/version1/spatial-reference-system-data.md are mis-labelled: Mx/Ux is the longitude and My/Uy is the latitude, correct?  So is 0.000020308287 the longitude offset or the y-offset (which is really latitude)?  Also, are you still assuming Mx/Ux are in UTM units here, or are these in degrees?  @tingli3's script seems to assume they are degrees; he applies the offsets after doing the proj4 projection.  If they really are degrees, could you clarify this in the documentation?

@gsrohde Thanks for pointing this out. I just fixed it. It was a typo. You are right. Mx/Ux is longitude in degrees; My/Uy is latitude in degrees. I have specified in the original doc that it was to convert from MAC lat,lon to USDA lat,lon.

@yanliu-chn Thanks for fixing.  I guess using the terms "lat" and "lon" makes it clear that in the MAC --> USDA (M2U) section we are no longer working in UTM coordinates.  What was confusing to me is that in the Gantry --> MAC section, the variable Mx is used to represent the easting coordinate in the UTM system whereas in the MAC --> USDA section, Mx is used to represent the longitude.  (Similarly for My.)

@dlebauer, @yanliu-chn I'm not sure which SQL statements I should be looking at.  The statements generated by the code in SQL.zip mentioned in comment https://github.com/terraref/computing-pipeline/issues/174#issuecomment-253279547 doesn't seem to directly relate to BETYdb, but @dlebauer references SQL statements designed for BETYdb in his comment https://github.com/terraref/computing-pipeline/issues/174#issuecomment-253669138.

@gsrohde I unassigned you when I realized more work needed to be done. But I went ahead and inserted these  ....
insert_season2_sites.sql.txt
for this case I am not worried about the incorrect use of now(), but now that I know I won't worry about it.

@dlebauer Please give me an example of the sql statement I should generate.

@tingli3 please see https://github.com/terraref/computing-pipeline/issues/174#issuecomment-253869609

@tingli3 ... but leave out the created_at and updated_at columns.

@gsrohde Can you give me an example sql?

@tingli3 The file David attached to the comment https://github.com/terraref/computing-pipeline/issues/174#issuecomment-253869609 has several example insertion statements.  Here's the first (but with the timestamp columns omitted, as they should be):
INSERT INTO sites (sitename, geometry) VALUES ( 'MAC Field Scanner Field Plot 1 Season 2',  ST_GeomFromText('POLYGON((-111.975049874375 33.0745312921391 353, -111.975033517034 33.0745313124814 353, -111.975033529346 33.0745670737771 353, -111.975049886694 33.0745670534346 353, -111.975049874375 33.0745312921391 353))', 4326));

season1_sql.txt
season2_sql.txt
Could someone please help me to verify these results?

@tingli3

I've already generated and inserted the season2 plots.
The season 1 plots should have update rather than insert statements; the sitenames do not include  Season 1` for the first season
please add your code to the github repository here: https://github.com/terraref/computing-pipeline/blob/master/scripts/geospatial/field_scanner_plots.R

@tingli3 issue #187 will check for consistency with other field measurements.

@dlebauer The season 1 and season 2 sql statements are based on the example given. If season 1 is different from season 2, please give me another example. Also, I have added the code.

@tingli3 I already inserted season 2
For season 1, the first line would be
UPDATE sites
  SET geometry =
    ST_GeomFromText('POLYGON((
      -111.975049873054 33.0745274579941 353,
      -111.975033515714 33.0745274783365 353,
      -111.975033525032 33.0745545428892 353,
      -111.975049882377 33.0745545225468 353,
      -111.975049873054 33.0745274579941 353))', 4326));
  WHERE sitename =
       'MAC Field Scanner Field Plot 1';

P.S. I reformatted your R code following http://adv-r.had.co.nz/Style.html so ... please pull changes before updating.

season1_sql.txt

@tingli3 all set - thanks! Closing - can you please update documentation (https://github.com/terraref/documentation/issues/7)?


Not operator syntax reccomendation

Negative Lookahead
~ expr

It would be a lot more intuitive, in my opinion at least, to use ! instead of ~. It seems like most modern languages these days use ! to mean "not". Just a recommendation.

Thanks for the suggestion, but ohm is not like "most modern languages" :p


[5.0.2] Use a route handler for the PDF rendering

[5.0.2] - 2024-06-08
Changed

Switch to using a route handler for PDF rendering

The /pages folder is finally gone 🎉

Resolves #509


Require developer API key for mapzen.js

Following on the discussion around issue #331, we've decided to move forward with requiring a Mapzen developer API key for any instantiation of mapzen.js.  This key will be used for all services requiring a Mapzen developer key (unless a separate key is provided in the specific service's options).
For now, recourse will simply be a console warning. See #331 for potential warnings (though certainly open to suggestions on improved wording.)

Closing: console warning added in #345


[Feature Request] Highlight plug-in when update is available

Material skin indicates that an update is available and, when selected, takes the user to the information screen and moves the display to the section with the update.
However, there is no indication which plug-in has been updated nor a method to see what will change after a reset.
I would like to see an indicator of what plug-in is ready for an update.  Methods might include changing the color of the plug-in text, adding an icon before or after the plug-in, or adding text with a hyperlink to open the v-sheet with plug-in detail

My mistake.
Many thanks.


Display the Android build number (e.g. MRA58K) in the app

This is so it's easy to record test results with a build number rather than having to go through the Android settings menus

Also added build number on the new About page in #58


Wireguard interface shown as IPv4 upstream instead of WAN interface

OpenWrt 18.06.0 x86_64
luci-ssl - git-18.212.55608-5ec7261-1
After configuring a Wireguard interface, Luci shows the Wireguard interface as IPv4 upstream, instead of the real WAN interface.
https://ibb.co/jVZqQp

LuCI shows the first interface carrying a default route as upstream interface.

But my default route is via eth1
root@OpenWrt:~# ip r
default via 90.162.x.x dev eth1 proto static src 90.162.x.y
<IP_ADDRESS>/24 dev eth1 proto kernel scope link src 90.162.x.y
<IP_ADDRESS>/24 dev eth0 proto kernel scope link src <IP_ADDRESS>
<IP_ADDRESS>/24 dev eth2 proto kernel scope link src <IP_ADDRESS>
<IP_ADDRESS>/24 dev eth3 proto kernel scope link src <IP_ADDRESS>
<IP_ADDRESS>/24 dev vpn proto kernel scope link src <IP_ADDRESS>

You are right, unchecking "Route Allowed IPs" in the WireGuard peer, and rebooting fixes the problem.
Thank you.


Add oauth2 owner flow support

This is work for #19.
@evert,
I've added a new auth type: oauth2 and support for the owner flow. What I've done so far is to just request the authorization token when it is missing and set it in the ketting object. You can see in the new tests how configuring this works. This makes use of https://github.com/mulesoft/js-client-oauth2 for managing the oauth client.
I still need to add support for refreshing the token and updating the docs, but I'd like to get your initial thoughts.

Just need to update the docs before this is completely ready to go.

K ! feel free to merge if you feel good about it, and let me know if you need me to publish a new release on npm.

Can you have a look at the updated REAME to make sure it makes sense and fits in with the overall flow and style?

Still looks great. Thanks for correcting my typos


Update curations/git/github/gradle/gradle.yaml

Just a heads up, this one is not going to show up.
This is one of the components suffering from size too large issues
see https://github.com/clearlydefined/service/issues/254


#2562 Distributor: Subscription topic not required anymore

Closes #2562
Co-authored by Godtide and @bacherfl

Codecov Report

Merging #2635 into master will increase coverage by 3.78%.
The diff coverage is 76.19%.

@@            Coverage Diff             @@
##           master    #2635      +/-   ##
==========================================
+ Coverage   33.31%   37.10%   +3.78%
==========================================
  Files         161      166       +5
  Lines        8891    10552    +1661
  Branches      185      185
==========================================
+ Hits         2962     3915     +953
- Misses       5429     6020     +591
- Partials      500      617     +117

Flag
Coverage Δ

moduleA
23.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

distributor/cmd/main.go
53.40% <76.19%> (+11.70%)
:arrow_up:

cli/cmd/delete_service.go
13.88% <0.00%> (-2.12%)
:arrow_down:

cli/cmd/install.go
12.98% <0.00%> (-1.90%)
:arrow_down:

helm-service/pkg/helm/generated_chart_handler.go
0.00% <0.00%> (ø)

.../pkg/configurationchanger/configuration_changer.go
0.00% <0.00%> (ø)

shipyard-controller/common/keptn_helpers.go
0.00% <0.00%> (ø)

shipyard-controller/api/service.go
34.96% <0.00%> (ø)

shipyard-controller/api/api_base.go
0.00% <0.00%> (ø)

shipyard-controller/common/secret_store.go
0.00% <0.00%> (ø)

shipyard-controller/api/project.go
36.73% <0.00%> (ø)

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4d19d58...3b6df3b. Read the comment docs.


How Chatto differentiate bubbles in group chat?

How can i differentiate bubbles during a group chat? I mean i want to show the name of sender in bubbles. Is this supported in current version?

@mohitaddval As I know, now only user's avatar is supported.

is there any update ?

+1


Fix DEV stack authentication url with localhost redirect_uri

I tried testing this. What I observed was:

I clicked "Sign in" in OSMCha
I was directed to osm.org to authorize OSMCha to access my account
After clicking "Authorize" I was directed back to OSMCha, but instead of being logged in, I see an error page that says Source authorized does not exist

Is that expected? In Slack you mentioned that there's a backend issue that affects this too, so maybe that's what I'm seeing, but just wanted to check.

@jake-low I think this solution will not work. I'm seeing this error in the osmcha backend when I try to login:
oauthlib.oauth2.rfc6749.errors.InvalidGrantError: (invalid_grant) The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client.

I was looking into this a bit yesterday. I think I understand the problem with this approach now, though I still need to think a bit more about how to solve it.
The InvalidGrantError is happening at the end of the authentication flow. The flow looks roughly like this:

the user clicks "sign in" on osmcha.org. this triggers an empty POST request to osmcha.org/api/v1/social-auth/. The backend replies with a an OAuth login URL to openstreetmap.org, and the frontend opens that page for the user. What this PR changes is that the frontend modifies the URL before opening it, changing the redirect_uri parameter set by the server.
the user clicks "authorize" on openstreetmap.org, which redirects them back to where they came from (more specifically, back to the specified redirect_uri, assuming it is on the approved list of redirect URIs registered for the application). An authorization code is included as a query parameter on this URL.
the frontend pulls the query parameter out of the URL and POSTs it to the backend (on the same osmcha.org/api/v1/social-auth/ endpoint as before, but with a body this time).
the backend attempts to use that code to fetch an access token from openstreetmap.org. this access token is what will allow the backend to interact with the openstreetmap.org API on the user's behalf. This is where the error occurs.

When requesting an access token, the backend must provide the authorization code to the OAuth provider (openstreetmap.org). But it must also provide the redirect URI, and it is an error if this redirect URI doesn't match what the frontend used when it generated the code (see RFC 6749 Section 4.1.3 for the specifics).
The frontend has tampered with the URL that the backend provided it, but the backend doesn't know this. By itself, this tampering is fine (anyone can generate these URLs; you don't need to know the application secret). The authorization flow succeeds, at least from the perspective of openstreetmap.org, and a valid authorization code is sent back to the frontend via redirect. But the backend doesn't know the correct redirect URI required to use this code. When it tries to use the code with the incorrect redirect URI to fetch an access token, openstreetmap.org returns an error.

The solution to this is to have the backend somehow know the correct redirect URI to use for a code, which on its own isn't too hard. For example, instead of using a fixed value from an environment variable, it could use the value of the Origin header of the HTTP request to the /api/v1/social-auth/ endpoint. That would allow different frontends hosted on different domains to use the same backend. (This is roughly what MapRoulette does to support running the frontend on localhost but having it talk to the staging or prod backend).
However, the requests-oauthlib package doesn't seem to be designed for this use case. It's OAuth2Session class takes the redirect_uri as an __init__ parameter, and doesn't allow modifying it later. You might be able to create a separate session object for each request, but I'm not sure if this is possible (there may be some state stored in the session that needs to match between the initial auth request and the callback request) or cryptographically safe.
Another possibility would be to forego the requests-oauthlib package and just use oauthlib directly.

For convenience, here is the full stack track of the error from the backend. I have no idea why the message at the end is in Czech. It translates to "The authorization grant provided is invalid, expired, revoked, does not match the redirect URI used in the authorization request, or was issued to another client".
Internal Server Error: /api/v1/social-auth/
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "/app/osmchadjango/users/views.py", line 111, in post
    access_token = self.get_access_token(
  File "/app/osmchadjango/users/views.py", line 87, in get_access_token
    return self.consumer.fetch_token(
  File "/usr/local/lib/python3.10/site-packages/requests_oauthlib/oauth2_session.py", line 406, in fetch_token
    self._client.parse_request_body_response(r.text, scope=self.scope)
  File "/usr/local/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 427, in parse_request_body_response
    self.token = parse_token_response(body, scope=scope)
  File "/usr/local/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 441, in parse_token_response
    validate_token_parameters(params)
  File "/usr/local/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 448, in validate_token_parameters
    raise_from_error(params.get('error'), params)
  File "/usr/local/lib/python3.10/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 399, in raise_from_error
    raise cls(**kwargs)
oauthlib.oauth2.rfc6749.errors.InvalidGrantError: (invalid_grant) Poskytnutý grant pro udělení oprávnění je neplatný, jeho platnost vypršela, zrušena, neodpovídá identifikátoru URI přesměrování použitého v žádosti o autorizaci nebo byla vydána jinému klientovi.

@jake-low feel free to try the solution you proposed, although I think this issue is not a high priority right now.


Make FocusMaxOrEqual work nicely with excluded filetype windows

Hey @beauwilliams thanks again for this awesome plugin,
I while ago you implemented a feature request I made for completly expanding windows, I've been using it daily. I have encountered one issue though, I'm using nvim-tree and toggleterm for openning terminal at the bottom.
Having either nvim-tree or terminal open if I do FocusMaxOrEqual to collapse back excluded windows won't be resized back to their initial size.
Two options come to mind which might be left configurable if possible:

Don't assume excluded windows as available screen space
Remember sizes of excluded windows when expanding.

This is my config:
  use { "beauwilliams/focus.nvim",
    config = function()
      require("focus").setup({
        excluded_filetypes = {'toggleterm', 'Trouble', 'NvimTree', 'dapui_scopes', 'dapui_breakpoints', 'dapui_stacks'},
        signcolumn = false
      })
      vim.cmd [[ FocusDisable ]]
    end,
    setup = function()
      vim.api.nvim_set_keymap("n", '<c-w>=', ':FocusMaxOrEqual <CR>', {})
    end
  }

This is a tough one. There really is not much that can be done to address this with built in vim features, as it involves using the standard vim command to equalize windows wincmd=
Now, remembering the sizes of excluded windows could be possible. It would require quite a bit of work, I actually had this issue earlier trying to implement a better resizing function, finding that there was simply a lack of API to remember widths of windows and set them later. Let me look into it

Another idea would be to just expose a callback willExpand or something to manually close excluded split windows like terminal and  file tree before maxing out the window.

Please try https://github.com/nvim-focus/focus.nvim/tree/refactor
Take a look at the Disabling Focus section:
https://github.com/nvim-focus/focus.nvim/tree/refactor#disabling-focus

Fixed in current master/main.


Support parsing json in MySql binglog

There is an API in mysql-connector-java that supports parsing the json in MySql binlog, so you can call it directly

and （ time、char ） field type

@barryoneill hello, Is it convenient to merge it?

Hello @caicancai - I'd prefer to have a little test coverage in place first, so I just raised https://github.com/laserdisc-io/mysql-binlog-stream/pull/392, then I'll merge this and update the test.

appears to have broken with the latest from master.  I'll pull it in anyway, and fix it up

thank you

v3.0.0-RC1 has just been published, and will available in maven central in a few hours.


Revert "PR testsuite should test like master"

Reverts uyuni-project/sumaform#1211
Hi ! This is not correct. This PR is adding a repo from opensuse mirror infrastructure:
baseurl: http://{{ grains.get("mirror") | default("download.opensuse.org", true) }}/repositories/systemsmanagement:/Uyuni:/Master:/SLE15-Uyuni-Client-Tools/SLE_15/
we are in purpose not using these repos in the uyuni-pr
We should use the repos from the worker instead:
See: https://github.com/SUSE/susemanager-ci/blob/master/jenkins_pipelines/environments/common/pipeline-pull-request.groovy#L250

The problem is, that the testsuite expect a repo with a special name. Just provide an alternative url under the same name and we are fine. But we need a 'tools_update_repo'.


wcslib: 7.6 -> 7.7

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 130459 at c6bd32b0 run on aarch64-linux 1

1 package marked as broken and skipped:

siril

3 packages built successfully:

kstars
stellarsolver
wcslib

Result of nixpkgs-review pr 130459 run on x86_64-linux 1

4 packages built:

kstars
siril
stellarsolver
wcslib


Better layout for basic/rounded/block buttons

Basic/Rounded/Block buttons should be split into tabs or something similar. This is necessary as the website is getting longer with every button added.

Does it mean to split buttons in Basic Buttons into several tabs like Animated Buttons? Can I work on this issue?

No it means that each button should have a tab where you choose to reveal how it looks like as basic, rounded or block

I tried to say the tabs like Bounce and Click under Animated Buttons tab. Did I understand it right?

No. Basically in Bounce button there would be tabs that says Basic, Rounded and Block and on clicking each it will show the Bounce button as basic or rounded or block. It doesn't have to be tabs if there's a better way of organizing it but that's the approach for now.

Oh I think I understood. Can I work on "Basic Buttons" section first and continue for the other sections after the confirmation?

Ok I will assign you.

@shahednasser @lee00286 I think one of the better ways is to just dynamically change the classes of the buttons when the respective tab-button is clicked. So you will just have add tab-buttons and rest should be taken care of by js.

@HADES-01 yes but with the current way at least the execution is done during loading time and no extra execution is needed. I'm afraid the second way might end up buggy.
Maybe at first we can try the easy way which is to just add the tabs and hide/show on click, later on if we find it more optimal to do it the way you mentioned then we can optimize it.

Hi @lee00286 how is this issue going? Do you need any assistance?

@shahednasser Hello I'm working on buttons tab that should display basic, rounded, and block buttons. I edited index.html to only show basic buttons at this point and each header (this is temporary decision for test... I will create buttons afterwards) directs to buttonsPage.html which I created.
Currently I am thinking about the way to show only relative buttons in buttonsPage.html without creating html file for each section.

Hi @lee00286 thank you so much for your work! Unfortunately I will have to close this issue as you didn't reply before and we changed this issue to add it to another workflow of this project. I'm sorry and again thank you for your contribution!


Strategy to extending IUrlHelper

What is the recommended strategy to extend IUrlHelper?
In previous version of MVC, there was public property RequestContext on UrlHelper allowing access to request info from within static extension method. Similar properties exist in current version of UrlHelper, but are all protected which prevents us from extending the helper and forcing us to use inheritance instead. This works when user wants to customize UrlHelper and inject it into his application, but denies extending UrlHelper from multiple locations (libraries).

Do you have a particular scenario for an extension method for IUrlHelper or UrlHelper within ASP.NET 5? Perhaps there is another way to achieve what you desire.

My scenario are the following helpers:

Is(object values) - check whether the current request contains given values
Has(params string[] keys) - check whether the current request contains given keys regardless of their values
Current(object with = null, string[] without = null) - returns current url with given values (with) and without given keys (without)

Note that all of these work on all of RouteData.Values, RouteData.DataTokens and Request.Query at the same time, so for path /users/index?page=10 Url.Current(with: new { sort = "name" }) would return /users/index?page=10&sort=name and not just /users/index?sort=name like Url.Action(null, new { sort = "name" }) would.
MVC 5 implementation: https://github.com/Kukkimonsuta/Clutch/blob/master/src/Clutch.Web.Mvc/CurrentUrlHelper.cs
MVC 6 implementation or Is:
public bool Is(object values)
{
	var actionContext = ActionContext;

	var query = actionContext.HttpContext.Request.Query;
	var routeValues = actionContext.RouteData.Values;
	var dataTokens = actionContext.RouteData.DataTokens;

	foreach (var pair in new RouteValueDictionary(values))
	{
		var value = pair.Value.ToString();

		if (query[pair.Key].Any(v => string.Equals(v, value, StringComparison.OrdinalIgnoreCase)))
			continue;

		var routeValue = default(object);
		if (routeValues.TryGetValue(pair.Key, out routeValue) && string.Equals(routeValue?.ToString(), value, StringComparison.OrdinalIgnoreCase))
			continue;

		var dataToken = default(object);
		if (dataTokens.TryGetValue(pair.Key, out dataToken) && string.Equals(dataToken?.ToString(), value, StringComparison.OrdinalIgnoreCase))
			continue;

		return false;
	}

	return true;
}

However this done by inheriting UrlHelper and injecting the new implementation instead, which effectively breaks any sort of similar extension.

One option is to write your own distinct helper instead of reusing/replacing IUrlHelper.
That is, just invent your whole new thing:
public interface ISuperUrlHelper
{
    bool SomeSuperMethod(...);
}

public class MySuperUrlHelper : ISuperUrlHelper
{
    public MySuperUrlHelper(IDependency1 dep1, IDependency2 dep2, ...)
    {
        ...
    }

    public bool SomeSuperMethod(...)
    {
        return dep1.Whatever + dep2.Whatever2;
    }
}

And then register this service in your app's Startup. And then in your controllers and/or views use your new super service.
Ultimately, the problem here is somewhat of a philosophical one: how far should you be able to extend an interface beyond its current limits? What if your method needs some completely random service like IFrobulator? That will never be added to UrlHelper or IUrlHelper because it is too specialized.
So, the philosophy here would indicate that if you're doing something that is beyond what IUrlHelper does then you probably don't belong on IUrlHelper.

Ultimately, the problem here is somewhat of a philosophical one: how far should you be able to extend an interface beyond its current limits? What if your method needs some completely random service like IFrobulator? That will never be added to UrlHelper or IUrlHelper because it is too specialized.

If the extensions needed to work with other services, that's probably where I'd draw the line, but I believe these helpers should be extensible within their intended use. IUrlHelper should expose ActionContext because it's reasonable to assume, that every implementation will depend on it and extensions will need it. The same works for IHtmlHelper already, which exposes ViewContext ( https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNet.Mvc.ViewFeatures/Rendering/IHtmlHelper.cs#L45 ).

IUrlHelper should expose ActionContext because it's reasonable to assume, that every implementation will depend on it and extensions will need it. The same works for IHtmlHelper already, which exposes ViewContext

This is something we could do. We already expose it as a protected property on the concrete class for scenarios where you subclass.

Yeah I agree we might as well expose it. Pretty useless otherwise.

Let's expose the context through the interface so that extension methods can use it.


Brackets checking removed from the return condition.

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Tests pass?
❌

Fixed issues
#18001

Let watch how the tests falls.

Seems to fail tests. At least table quoting ones.

I updated the code to check brackets in first and last position in table name string. When all tests passed I also added new test to check quoting table name with brackets.

@samdark Sorry for my stupid question, but its my first contribute in a such big project. So, my question is about CHANGELOG. I made all this changes from the last stable release 2.0.35  (because master branch had test fails at the moment of forking). So, my version of CHANGELOG hasn't list of 2.0.36 changes. And im not sure that it will be merged properly if I just add new line. So, I should merge master into my branch, add the line with my changes, and make new pull request, o something else?
Thanks.

@floor12 I'll handle it, don't worry.

Merged. Thank you.


Weird launching Godot from Steam error [Invalid mapping entry: dpright]

Godot version
3.3.3.stable
System information
Windows 10
Issue description

The error shows up when I just launch Godot using Steam.
Launching Godot from the folder doesn't cause the error.
Reinstall doesn't help either.
Steps to reproduce
I launch my Godot from the steam launcher.
Minimal reproduction project
I have no idea what to put in, so I was going to put my whole Godot file without my projects in here, but Github doesn't allow me so. ¯_(ツ)_/¯

Do you have a controller connected to the PC when Godot is launched? If so, which controller model is it?

This might happen if your Steam config includes an invalid custom gamepad mapping, which is then passed to all games (including Godot) with the SDL_GAMECONTROLLERCONFIG environment variable. (Godot doesn't use SDL but uses its gamepad mapping format, including this config environment variable.)
For example here on Linux it passes:
SDL_GAMECONTROLLERCONFIG=03000000de280000ff11000001000000,Steam Virtual Gamepad,a:b0,b:b1,back:b6,dpdown:h0.4,dpleft:h0.8,dpright:h0.2,dpup:h0.1,guide:b8,leftshoulder:b4,leftstick:b9,lefttrigger:a2,leftx:a0,lefty:a1,rightshoulder:b5,rightstick:b10,righttrigger:a5,rightx:a3,righty:a4,start:b7,x:b2,y:b3,

(I could find out by editing the run command to env && %command% on Linux and running Steam in a terminal to see the console output, I'm not sure what could be the equivalent on Windows.)
Wrong mappings might be saved in your Steam config file, see https://www.reddit.com/r/SteamController/comments/8qyhs5/want_to_reset_steams_controller_settings_heres_how/ for instructions on how to find it and see if there's anything fishy (especially a mapping with dpright:, and no h0.2 or similar following it) on the SDL_GamepadBind line.

Hmm, I followed the instruction but I seem to can't find SDL_GamepadBind store in Steam's config folder.

That's right! There's nothing in this variable

Thanks! This does fix the problem.


chore: update namespace

Should we make an upstream PR for this? We can use that PR commit until it's merged. Or do you think we should fork it with this repo?

@mertalev we can upstream it but I think this is not ready yet


Create stack.yaml

As this is not a multi-repository project I don't see any advantage (and several disadvantages) in having a stack.yaml file here.
If you wish to use stack when developing this on your own then you can have it locally.

No problem.
BTW, thanks for the quick response.


Calculations using ovuli library

A user will now be able to see Approximate Ovulation date, Next Period Date, Next Pregnancy Date and Fertile Window. These calculations are being done using the calculateOvuli function present in the ovuli library

Closes #65. Please review @sarthology

@Rits1272 Gif for it

Please check @sarthology

Fabulous Speed Champ !!


Restore two more instances of CountableRange

Similar to #1423 to keep compatibility with both the 5.0 branch and master. To be reverted once swift rebranches 5.0.

@swift-ci test


Add a builtin implementation for the pwd shell-command

Codecov Report

Merging #8 into master will decrease coverage by 0.58%.
The diff coverage is 88.14%.

@@            Coverage Diff             @@
##           master       #8      +/-   ##
==========================================
- Coverage   89.53%   88.94%   -0.59%
==========================================
  Files         139      146       +7
  Lines        9118     9446     +328
==========================================
+ Hits         8164     8402     +238
- Misses        954     1044      +90

Impacted Files
Coverage Δ

tests/support.rs
91.07% <ø> (ø)
:arrow_up:

src/lib.rs
100% <ø> (+45%)
:arrow_up:

src/spawn/builtin/true_cmd.rs
77.77% <100%> (ø)
:arrow_up:

src/spawn/builtin/false_cmd.rs
77.77% <100%> (ø)
:arrow_up:

src/spawn/builtin/shift.rs
82.92% <100%> (-1.86%)
:arrow_down:

src/spawn/builtin/colon.rs
77.77% <100%> (ø)
:arrow_up:

src/env/cur_dir.rs
87.27% <100%> (-3.64%)
:arrow_down:

tests/exit_status.rs
36.66% <100%> (ø)

src/spawn/builtin/mod.rs
100% <100%> (ø)

src/exit_status.rs
60% <60%> (ø)

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 50619f5...14c7f31. Read the comment docs.


Fixes DF-1963 - subitems in sidenav

Fixes DF-1963 - subitems in sidenav. Adds inset to sidenav sub-pages to
visually distinguish them from sibling pages.
Additions

Added .nav-secondary_link__disabled style.
Added .nav-secondary_subitem class to visually distinguish sub-pages from sibling pages in the sidenav.

Changes

Updates megamenu bureau title to "The Bureau" to fit with sitemap direction.
Removes extraneous whitespace from small businesses page.
Regenerates CSS assets.

Testing

Visit /the-bureau/ or other pages that have subpages and note that subpage is inset in the sidenav.

Review

@sebworks
@KimberlyMunoz
@jimmynotjim

Preview

Notes

There should be no visual change at mobile and tablet sizes, however the classes between the parent and subpage links are different (nav-secondary_item and nav-secondary_subitem, respectively).

:+1:


Local Testnet Deployment

Deploy a long-lived local testnet to simulate the complete flow of consensus through Lotus instances. Ensure correct integration with Gossipsub and other components.

This is now unblocked when running https://github.com/filecoin-project/lotus/pull/12119


automatically generated  not support

To Reproduce
Steps to reproduce the behavior:
Which Python version are you using?
Which version of youtube-transcript-api are you using?
What code / cli command are you executing?
Expected behavior
Describe what you expected to happen.
Actual behaviour
Describe what is happening instead of the Expected behavior. Add error messages if there are any.

@lengrongfu please properly describe what is not working as expected using the template I inserted.

@jdepoix Hi, Youtube translated subtitles to be downloaded?
e.g.
transcript_list = YouTubeTranscriptApi.list_transcripts(video_id="8iYCp3ZwoU8")
print(transcript_list)
For this video (8iYCp3ZwoU8) transcripts are available in the following languages:

(MANUALLY CREATED)
None

(GENERATED)
 - en ("English (auto-generated)")[TRANSLATABLE]

(TRANSLATION LANGUAGES)
 - af ("Afrikaans")
 - ak ("Akan")
 - sq ("Albanian")
 - am ("Amharic")

but transcript = YouTubeTranscriptApi.get_transcript(video_id="8iYCp3ZwoU8", languages=['af']) donot download.

This is correct since there is no (manually nor automatically generated) transcript in Afrikaans. There only is an automatically generated English transcript, which can be translated into Afrikaans. To do so, please have a look at the README.
I will close this now, as this seems to work as intended.


Feedback Wanted: Unit Testing

What is the current state of Unit Testing? Is it design-flowed so far?
https://www.benthos.dev/docs/configuration/unit_testing

I'm fairly happy with it but there are some minor adjustments that might be worth considering. Let's aim to stabilize it in the coming months. I'll use this issue to throw ideas around.
Stuff worth considering:

Do we want to keep the test filename as <target>_benthos_test.yaml? I'm thinking it's a pain and we can get away with <target>_test.yaml
Do we want to consider allowing tests within the target config?

Unit testing got a lot of love, I'll remove the EXPERIMENTAL flag from the docs next time I update it but I don't plan any future changes. The only optional improvement I might make is to be able to define tests within the target config itself, but I'm not convinced it would be a popular option as the test definitions very quickly become so large it wouldn't be practical.


Add failing test for font-face

To help to trace down #42

@rickharrison Have you had a chance to run this tests and see the failure? Just to confirm that is a legitimate bug and not that I'm doing something wrong.

So for me this test is passing. the only change I had to make was to remove the #robotoregular from the outputted css. I think you might have an issue in your broccoli definition. Could you post how you are trying to use broccoli asset rev and your folder structure?

I'm going to close this for now. Please let me know if you figured out your issue or not.

I forgot about this. I figured it out. If I recall correctly the problem was that I was prepending the url with a /, but I'm not 100% sure I'm telling you the truth.
Thanks!

Cool, glad its all worked out!


Implement bindings for StoreContext data

closes #190
This PR adds functionality to the Store and Caller that allows users to store arbitrary data in the WASM Store and access it using host functions like so:
var storeData = "Hello, World";
var engine = new Engine();
var linker = new Linker(engine);
var store = new Store(engine, storeData);

linker.DefineFunction("", "store_data", (Caller caller) => {
    var data = caller.GetData() as string;
    Console.WriteLine(data);
});

TODO:

[x] Add binding for wasmtime_context_get_data

[x] Add binding for wasmtime_context_set_data

[x] Add storedata example

[x] Add test for SetData()

Thanks for the feedback @kpreisser !
I'll fix the failing test case (dumb mistake) and add an example to the funcrefs later today or tomorrow. Anything else you can think of? Are the two testcases sufficient?

Added an example and actually working testcases together with the binding for wasmtime_context_set_data. Think it's functionally finished right now.

@Yeseh thanks again for implementing this!

@peterhuene just noticed that we all missed the storedata example being in the root folder of the repo instead of examples/ :)

Ah, how little one pays attention to a path when there's fun code to review 😅.
Thankfully the repo layout doesn't matter much and we can address it easily with a follow up.


Tests: testThaliMobileNative - Can shift data securely fails

The issue with this test occurs because we close socket on client's side, but we should do it on the server's side. In that case, close event will be triggered on the client's side in proper way, time.
In this logs we can see that the teardown is already called, but server is still receiving data.
Connected to the localhost:50177
2017-06-30 12:53:39 - DEBUG testThaliMobileNative: 'Client sends data (20 bytes):'
2017-06-30 12:53:39 - DEBUG testThaliMobileNative: 'Client data flushed'
ok 117 got the same data back
[ThaliCore] TCPListener.socketDidDisconnect(_:withError:) accepted socket error:Optional(Error Domain=GCDAsyncSocketErrorDomain Code=7 "Socket closed by remote peer" UserInfo={NSLocalizedDescription=Socket closed by remote peer})
[ThaliCore] BrowserRelay.didSocketDisconnectHandler
[ThaliCore] VirtualSocket.closeStreams() vsID:9
[ThaliCore] BrowserRelay.didCloseVirtualSocketStreamsHandler
# teardown
[ThaliCore] Session.session(_:peer:didChange:) peer:3CA8DCA5-8F44-4E7B-90E1-DF79C95E1B17:0 state: connected -> notConnected
[ThaliCore] Browser: session notConnected Peer(uuid: "3CA8DCA5-8F44-4E7B-90E1-DF79C95E1B17", generation: 0)
[ThaliCore] BrowserRelay.closeRelay() disconnecting:false
[ThaliCore] TCPListener.stopListeningForConnectionsAndDisconnectClients() port:50177
[ThaliCore] TCPListener.socketDidDisconnect(_:withError:) listening socket error:nil
[ThaliCore] Session.disconnect() peer:3CA8DCA5-8F44-4E7B-90E1-DF79C95E1B17:0
[ThaliCore] Browser: session notConnected removed relay for Peer(uuid: "3CA8DCA5-8F44-4E7B-90E1-DF79C95E1B17", generation: 0)
[ThaliCore] Session.session(_:didReceiveCertificate:fromPeer:certificateHandler:) peer:E7B5FDC2-FF6E-4738-9597-F8142D282574
[ThaliCore] Session.session(_:peer:didChange:) peer:E7B5FDC2-FF6E-4738-9597-F8142D282574 state: connecting -> connected
2017-06-30 14:53:41.866122+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:41.917644+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:41.956391+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
[ThaliCore] Session.session(_:didReceive:withName:fromPeer:) peer:E7B5FDC2-FF6E-4738-9597-F8142D282574
[ThaliCore] Session.startOutputStream(with:) peer:E7B5FDC2-FF6E-4738-9597-F8142D282574
[ThaliCore] VirtualSocket.init(inputStream:outputStream:) vsID:10
[ThaliCore] TCPClient.connectToLocalhost(onPort:)
2017-06-30 14:53:41.966297+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.012112+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.042999+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.067519+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.103559+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.147140+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.159798+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.196481+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.310072+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.335508+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.398299+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.442914+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 14:53:42.547458+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
Server received psk id: psk-id
2017-06-30 14:53:42.753451+0200 ThaliTest[279:12868] [ViceroyTrace] [ICE][ERROR] 	Send BINDING_REQUEST failed(C01A0041).
2017-06-30 12:53:42 - DEBUG testThaliMobileNative: 'Server received (20 bytes):'
2017-06-30 12:53:42 - DEBUG testThaliMobileNative: 'Server received all data: small amount of data'
2017-06-30 12:53:42 - DEBUG testThaliMobileNative: 'Server sends data back to client (20 bytes): '
2017-06-30 12:53:42 - DEBUG testThaliMobileNative: 'Server data flushed'
2017-06-30 12:53:42 - DEBUG testThaliMobileNative: 'Server's socket stream finished'
[ThaliCore] TCPClient.socketDidDisconnect(_:withError:) disconnecting:false socket error:Optional(Error Domain=GCDAsyncSocketErrorDomain Code=7 "Socket closed by remote peer" UserInfo={NSLocalizedDescription=Socket closed by remote peer})
[ThaliCore] TCPClient.socketDidDisconnect(_:withError:) client disconnected
[ThaliCore] AdvertiserRelay.didSocketDisconnectHandler disconnecting:false
[ThaliCore] AdvertiserRelay.disconnectNonTCPSession()
[ThaliCore] Session.disconnect() peer:E7B5FDC2-FF6E-4738-9597-F8142D282574
[ThaliCore] Session.deinit peer:E7B5FDC2-FF6E-4738-9597-F8142D282574
[ThaliCore] VirtualSocket.closeStreams() vsID:10
[ThaliCore] AdvertiserRelay.didCloseVirtualSocketStreamsHandler disconnecting:false
[ThaliCore] VirtualSocket exited RunLoop vsID:10
[ThaliCore] VirtualSocket.deinit vsID:10

The issue is most probably due some problems in Coordinated tests framework or in communication between server and peers, because it can fail on any other test. It could be also because of large amount of tests that have to be loaded. When running only one test file, such issues doesn't occur.
Closing for now.

So the failing scenario looks like this:
Client connects to server, sends data. Server receives data and sends it back, but in the same time we lose connection. That causes Client to not receive any data, Client socket won’t be closed and shiftData won’t resolve.
Sometimes it occurs on one peer, sometimes on both.
In fact, this scenario can occur in every test that uses shiftData function and wait for it to resolve when Client receives all data back from server
.
So it would be Can shift data, Can shift data via parallel connections, Can shift data securely, Can shift large amounts of data.
This problem is more general, so we should decide if we want to fix it or not. The potential fix would to be refactor these tests to be promises and control them from executeZombieProofTest with some timer and/or retry logic.


CTRL+C causes crash on mac

Hi.
When running robyn with python3 app.py causes python to crash and generate mac os bug report.
fish: Job 1, 'python3 app.py' terminated by signal SIGABRT (Abort)

Hey @luqasz 👋
Thanks for the issue. Could you share the code snippet in your app.py?

hey @luqasz , ctrl + c is supposed to kill the program 😅  What do you expect to happen here?

hey @luqasz , ctrl + c is supposed to kill the program 😅 What do you expect to happen here?

Gracefull shutdown. When I wrap fastapi app with KeyboardInterrupt I can intercept it and shutdown gracefully. With robyn I can't.

@luqasz , I see. So you expect an exception to surface, which you can handle.
This is definitely a bug. Will try to address it

Thx.

Hey @luqasz 👋
This has been fixed now 😄


Redundant nonce management

You could simply build a nonce method based on time, lets say:
    # This value is used in order to generate a small nonce,
    # because for some reason btc-e doesn't like big nonces.
    timestamp_reduce_value =<PHONE_NUMBER>200
    return int(time.time()*1e3) - timestamp_reduce_value

Apologies for not getting to this (much, much) earlier...I've been entirely too busy for a long time. :)
I'm not a security person, but I suspect doing something like this with the nonce creates an opportunity for an adversary with your keys to sneak in transactions without you being alerted, as it introduces gaps into the sequence of nonces you use.  Using a contiguous sequence of nonces makes it immediately apparent that someone else is using your keys as soon as you attempt to perform an operation after they choose a nonce you haven't used yet.


nispor: Introduce nispor plugin

Introduce NisporPlugin for querying on these informations using
python3-nispor package:

MTU, MAC, state, IP address.
DHCP/AUTOCONF status:

Mark DHCP/AUTOCONF as true when found IP address with valid_lft not
forever and correct prefix length.

As nispor might be wrong on DHCP status and interface type comparing to
NetworkManager, so NetworkManager plugin take priority over nispor
plugin and override conflict data.
Removed the code in NetworkManager plugin for querying above information
except the DHCP/Autoconf and IP stack enable status.
Integration test case added.
Signed-off-by: Gris Ge<EMAIL_ADDRESS>

Patch ready for review.

Wait PR #1278


Windows build failing after running C:\image\Tests\Wix.Tests.ps1

Description
azure-arm.vhd: Running tests from 'C:\image\Tests\Wix.Tests.ps1'
azure-arm.vhd: Describing Wix
azure-arm.vhd:   [+] Wix Toolset version from registry 7ms (3ms|4ms)
azure-arm.vhd: Tests completed in 250.58s
azure-arm.vhd: Tests Passed: 490, Failed: 0, Skipped: 13 NotRun: 0
==> azure-arm.vhd: Provisioning with Powershell...
==> azure-arm.vhd: Provisioning with powershell script: C:\Users\ADM~1.VSI\AppData\Local\Temp\powershell-provisioner1425946189
==> azure-arm.vhd: Provisioning with Powershell...
==> azure-arm.vhd: Provisioning with powershell script: C:\Users\ADM~1.VSI\AppData\Local\Temp\powershell-provisioner3906116939
==> azure-arm.vhd: ←[31;1mException setting "Version": "The argument is null or empty. Provide an argument that is not null or empty, and then try the command again."
==> azure-arm.vhd: At C:\image\SoftwareReport\software-report-base\SoftwareReport.Nodes.psm1:164 char:9
==> azure-arm.vhd: +         $this.Version = $Version
==> azure-arm.vhd: +         ~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm.vhd: + CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordException
==> azure-arm.vhd: + FullyQualifiedErrorId : ExceptionWhenSetting←[0m
==> azure-arm.vhd: Script exited with non-zero exit status: 1. Allowed exit codes are: [0]
==> azure-arm.vhd: Step "StepProvision" failed
==> azure-arm.vhd: [c] Clean up and exit, [a] abort without cleanup, or [r] retry step (build may fail even if retry succeeds)?
Platforms affected

[X] Azure DevOps
[ ] GitHub Actions - Standard Runners
[ ] GitHub Actions - Larger Runners

Runner images affected

[ ] Ubuntu 20.04
[ ] Ubuntu 22.04
[ ] macOS 11
[ ] macOS 12
[ ] macOS 13
[ ] Windows Server 2019
[X] Windows Server 2022

Image version and build link
20230820.1.0
Is it regression?
No
Expected behavior
Expect the build to move on
Actual behavior
azure-arm.vhd: Running tests from 'C:\image\Tests\Wix.Tests.ps1'
azure-arm.vhd: Describing Wix
azure-arm.vhd:   [+] Wix Toolset version from registry 7ms (3ms|4ms)
azure-arm.vhd: Tests completed in 250.58s
azure-arm.vhd: Tests Passed: 490, Failed: 0, Skipped: 13 NotRun: 0
==> azure-arm.vhd: Provisioning with Powershell...
==> azure-arm.vhd: Provisioning with powershell script: C:\Users\ADM~1.VSI\AppData\Local\Temp\powershell-provisioner1425946189
==> azure-arm.vhd: Provisioning with Powershell...
==> azure-arm.vhd: Provisioning with powershell script: C:\Users\ADM~1.VSI\AppData\Local\Temp\powershell-provisioner3906116939
==> azure-arm.vhd: ←[31;1mException setting "Version": "The argument is null or empty. Provide an argument that is not null or empty, and then try the command again."
==> azure-arm.vhd: At C:\image\SoftwareReport\software-report-base\SoftwareReport.Nodes.psm1:164 char:9
==> azure-arm.vhd: +         $this.Version = $Version
==> azure-arm.vhd: +         ~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm.vhd: + CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordException
==> azure-arm.vhd: + FullyQualifiedErrorId : ExceptionWhenSetting←[0m
==> azure-arm.vhd: Script exited with non-zero exit status: 1. Allowed exit codes are: [0]
==> azure-arm.vhd: Step "StepProvision" failed
==> azure-arm.vhd: [c] Clean up and exit, [a] abort without cleanup, or [r] retry step (build may fail even if retry succeeds)?
Repro steps
Clone the code and try building a windows VM.
I tried two times and failed exactly at same spot.

@Alexey-Ayupov  any help please?

We also started experiencing this error since 2 days ago.

Yes us too

We also encounter this problem and in an effort to localise the problem added extra logging in the constructor of the ToolVersionNode class in the SoftwareReport.Nodes.psm1 file
    ToolVersionNode([String] $ToolName, [String] $Version): base($ToolName) {         if ($null -eq $Version -or $Version -eq "") {             Write-Host "##vso[task.logissue type=error] NO version - ToolVersionNode: $($Toolname) : $($Version)"         }         $this.Version = $Version     }
The problem originates from "Ant" having no version. From the file SoftwareReport.Generator.ps1
$projectManagement.AddToolVersion("Ant", $(Get-AntVersion))
This file is called by packer as one of the providers - see either windows2019.json or windows2022.json:
        {             "type": "powershell",             "inline": [                 "pwsh -File '{{userimage_folder}}\\SoftwareReport\\SoftwareReport.Generator.ps1'"             ],             "environment_vars": [                 "IMAGE_VERSION={{user image_version}}"             ]         },
Not sure how to proceed from here though.. any help would be nice.

Hello, it does not fail wix toolset, it is failed due to \SoftwareReport.Nodes.psm1. It has already been fixed here https://github.com/actions/runner-images/pull/8139, please always sync your main before attempting any new builds.

Thanks @mikhailkoliada , i synced it last night and #8139 also failed but with a different error. after creating a storage acct it failed. Going to start a new session and try it.

@mikhailkoliada now i get image state undeployable with #8139 . DO you want me to open a new issue? can you reopen this?

@mikhailkoliada now i get image state undeployable with #8139 . DO you want me to open a new issue? can you reopen this?

any help please @Alexey-Ayupov ?


log-backup: PiTR restore failed due to restored_ts > log backup range

Dup https://github.com/pingcap/tidb/issues/36246

/assign
/feature developing

/component migrate

/severity Critical

/sig migrate

/severity Moderate

/type bug


What is the process for writing back from the step definition to the feature file?

Current behavior
Since we are integrating an API with UI automation, passing variable names in the feature file datatable . However, we require that the report should display the actual values that are returned from the API and be validated in the step definition.
Feature Description
Scenario: Log in to the app
Given User is logged in to the application
When I enter the following user information:
| Name   | Email              |
| <name> | <email_placeholder>|

Current Report
When I enter the following user information:
| Name   | Email              |
|  | <email_placeholder>|
Desired behavior
Desired cucumber html report
When I enter the following user information:
| Name   | Email              |
| abc    |<EMAIL_ADDRESS>
Test code to reproduce
When('I enter the following user information:', function (dataTable) {
dataTable.hashes().forEach((row) => {
let name = "abc"; // Hardcoded value for name
let email =<EMAIL_ADDRESS>// Hardcoded value for email
cy.wrap(email).as(actualEmail_${name});
cy.wrap(name).as(actualName_${name});
cy.get('@dataTableRows').then((rows) => {
cy.log("step def -1 ");
rows.push({
name: name,
email: email,
actualValue: '42' // Set the actual value here
});
});
});
});

Versions

Cypress version:
"@badeball/cypress-cucumber-preprocessor": "^12.0.0",
"@bahmutov/cypress-esbuild-preprocessor": "^2.2.0",
"cucumber-html-reporter": "^6.0.0",
"cypress": "^12.11.0",

Checklist

[x] I've read the FAQ.
[x] I've read instructions for logging issues.
[x] I'm not using<EMAIL_ADDRESS>(package name has changed and it is no longer the most recent version, see #689).

This isn't a support forum, but FYI what you're talking about doesn't make any sense.


There is a typo in the repo name, I assume you meant to write learning-kit

Thanks @whoiskekeanyway , I just corrected it.