Sub-classes of StatisticalDataset

Decided during May 7 meeting: define what sub-classes of StatisticalDataset we want.
Example candidate: TimeSeries.
Linked to issue #6.

Event History?

Checking GSIM v1.2 Dataset
Definition: An organized collection of data.
Explanatory text: Examples of Data Sets could be observation registers, time series, longitudinal data, survey data, rectangular data sets, event-history data, tables, data tables, cubes, registers, hypercubes, and matrixes. A broader term for Data Set could be data. A narrower term for Data Set could be data element, data record, cell, field.

There was an ADMIN VIP project in the EU, named ADMIN. That is what I found for data classification:
https://ec.europa.eu/eurostat/cros/content/statistical-data_en
In my opinion we should take into consideration only those statistical data classifications (from point of view of aggregation or level of process e.g) which can be used as parameters for IT application later on handling those kind of data. This parameter can trigger different kind of tasks for different kind of data sets in an IT application.

I remember that pre-GSIM we had classifications, but while creating GSIM it was pointed out that our classifications are more strict than other classifications ( its elements are mutually exclusive and complete) so we were persuaded to call the GSIM information object Statistical Classification.
I am not convinced that our datasets are any different from anyone elses datasets. I do not see an advantage in creating a specialisation. Our GSIM Datasets do inherit from Identifiable Artefact so maybe that is an essential difference?
Maybe we could just use themes or domains to say that our datsets are within statistics? The same could be done for almost all GSIM information objects so that we do not need to put 'Statistical' in front of them all. Or maybe we could use a name space gsim: ?

What do our ModernStats models say about statistical dataset (or dataset in general)?
GSBPM
Does not mention data sets. Basically it doesn’t have to. It’s a process model.
GSIM
•	Data Set: An organized collection of data. Examples of Data Sets could be observation registers, time series, longitudinal data, survey data, rectangular data sets, event-history data, tables, data tables, cubes, registers, hypercubes, and matrixes. A broader term for Data Set could be data. A narrower term for Data Set could be data element, data record, cell, and field.
•	Unit Data Set: A collection of data that conforms to a known structure and describes aspects of one or more Units. Example: A synthetic unit record file is a collection of artificially constructed Unit Data Records, combined in a file to create a Unit Data Set. Synonyms: Micro data, unit data, synthetic unit record file
•	Dimensional Data Set: A collection of dimensional data that conforms to a known structure.
•	Information Set: Organized collections of statistical content. Statistical organizations collect, process, analyse and disseminate Information Sets, which contain data (Data Sets), referential metadata (Referential Metadata Sets), or potentially other types of statistical content, which could be included in additional types of Information Set.
GAMSO
Does not mention datasets. It doesn’t have to as data sets are information objects.
CSDA
Despite the fact that we do not take CSDA into consideration there is a classification for data sets in the document, which is as follows:
•	Explorative: Data that is obtained from outside sources, is usually “sampled” and is used to assess the nature, structure and quality (usability) of that data source. After the exploration, this data in most cases loses its value.
•	Organizational: The true (data) assets of the organization, that are to be treated as such and must be protected and shared where possible. An important sub-type of “Organizational” is the Master Data such as statistical registers, back-bones of populations, collections of statistical units. For instance: Company register, People Register, Buildings register.
•	Temporary, local: Data that is produced as an intermediate product in a statistical process and has no real value outside that process. This data usually loses its value after the process (cycle) is completed, but may have value for the next cycle as a reference. May be persisted within the process space

Conclusion and suggestion:
Only such classification should be taken on board which are relevant from the point of view of information management meaning that these kind of datasets have to be handled in a different way (other process, other methods etc.). The sub-classes should also has a statistical perspective to make it easy for the user (statistician) of the ontology to understand the sub-classes we define.
GSIM provides one kind of classification (according to structure) for data sets, which is
•	Unit data set
•	Dimensional data set
This also corresponds to the breakdown of microdata/tabular (aggregated) data. We think that this breakdown provided by GSIM is a good basis. If we need further breakdown but we need to agree on the purpose.
•	We can differentiate unit data set by source (data collection, data transmission, other / unimode, multimode, etc.) for by phases where the dataset is made available (corresponding to GSBPM Phases IV, V. VI, VII).
•	We can classify dimensional data set by the type of data included (Nominal, Ordinal, Discrete, Continuous), by domains, or sensitivity (SDC perspective).
Maybe we can move forward with the currently available two sub-classes defined by GSIM. To be honest, this more like a feeling than a well-based opinion…

Datasets can be classified in so many dimensions that's really hard to come up with a comprehensive list. Summing up the postings above, some of them are:

scope: explorative, organizational, local (as per CSDA)
domain: social, economics, etc.
granularity: micro (unit), aggregate (dimensional)
sensitivity: privacy/confidentiality related (public vs. confidential? some scale from 1 to n?)

I'd add also

status: preliminary, edited/imputed, final, revised, etc.

I'm not sure how to classify nominal, ordinal, discrete, continuous because I don't know what the first two mean, and time series, longitudinal, event history either... Types of data?

The Code Lists used in DCAT are available in EU Vocabularies:
Relevant for us:

Theme (13 themes used for classifying datasets in EU and European open data portals)
Access right (Sensitivity)

A code list CL_CONF_Status is available in the SDMX Global Registry.

Sometimes I struggle to find the human readable content under EU Vocabularies.
A hint is to click on the blue button Browse content on the right hand side after you have chosen your vocabulary.
For dataset type that takes you to https://op.europa.eu/en/web/eu-vocabularies/concept-scheme/-/resource?uri=http://publications.europa.eu/resource/authority/dataset-type
For access rights that takes you to https://op.europa.eu/en/web/eu-vocabularies/concept-scheme/-/resource?uri=http://publications.europa.eu/resource/authority/access-right

ONS has a data sensitivity model and a content sensitivity model. In Statistics Norway we have adopted 4 levels of Privacy from the ONS model: private, confidential, commercial, open. ONS also had a Sensitivity Assessment tool that we have translated to Norwegian and are implementing in our organisation as part of our GDPR compliance.

These are all interesting comments. I think Flavio is on the right track. He comments there are many dimensions by which to create subtypes of datasets. I propose we follow the definition of dataset from GSIM - organized collection of data - and build subtypes based on organization. Other criteria for identifying subtypes are not germane from that point of view, and I comment below on why I think we should not base subtypes on them.
In DDI-CDI, 4 basic structural types of organizing data sets have been defined: rectangular, event history, key-value pair, and dimensional. Several of the types could be used to structure the same data. There is not a canonical structure in all cases, though some data is much more amenable to one structure over the others.
The types are defined roughly as follows:

rectangular (or wide) - rows are units and columns are variables
event history (or tall or long) - rows are based on the value for each variable, one unit at a time - and this could be visualized as rows are variables and columns are units
dimensional - a pre-defined set of cells defined by the combination of categories, one from each of a set of dimensions (category sets), used to handle the value of some measure (variable) restricted to the cell
key-value - a set of values, each associated with some key

Dimensional data are usually associated with aggregates. Key-value data are often taken from scraping the web. Even-history is used to describe events over some time period.
The nominal, ordinal, interval, ratio are not used to differentiate datasets. Rather, they are families of datatypes used to describe variables. Nominal data are those conforming to a finite set of categories with no other conditions (sex categories). Ordinal data are those conforming to an ordered finite set of categories, but the difference between adjacent categories is not necessarily uniform (Likert scale measures of satisfaction). Interval data are numeric with no zero (absence of quantity) defined (Celsius temperature). Ratio data are numeric with a defined zero (Kelvin temperature). These apply to any kind of statistical data.
The distinction between aggregate and unit data is based on the definition of the variables in the dataset. A dataset can contain both unit and aggregate data.
Access restrictions on data (e.g., public, restricted, private) are assigned by the business and can change over the life-cycle of the dataset.
The domain for a dataset is defined by the subject field that data apply to. However, some datasets are merged from others, so a merged set can have the combination of its constituents. There seems to be no restriction on the number of subject fields.
Mode of transmission is not definitional for a dataset, as a single dataset can be obtained multiple ways. The phases of GSBPM may not be useful, as a single dataset can pass through a phase without change. Further, the phases impose a usage criterion (data for collection; data for editing; etc.) that seems arbitrary and would be useless in another domain (outside statistics).
Similarly, the explorative, temporary, and organizational categorization is based on intent, rather than the data per se. Plus, the categorization could change without any change to the data. If we change the organizational structure described above (rectangular, etc.), then we should call that a new dataset.

These are all interesting comments. I think Flavio is on the right track. He comments there are many dimensions by which to create subtypes of datasets. I propose we follow the definition of dataset from GSIM - organized collection of data - and build subtypes based on organization. Other criteria for identifying subtypes are not germane from that point of view, and I comment below on why I think we should not base subtypes on them.
In DDI-CDI, 4 basic structural types of organizing data sets have been defined: rectangular, event history, key-value pair, and dimensional. Several of the types could be used to structure the same data. There is not a canonical structure in all cases, though some data is much more amenable to one structure over the others.
The types are defined roughly as follows:

rectangular (or wide) - rows are units and columns are variables
event history (or tall or long) - rows are based on the value for each variable, one unit at a time - and this could be visualized as rows are variables and columns are units
dimensional - a pre-defined set of cells defined by the combination of categories, one from each of a set of dimensions (category sets), used to handle the value of some measure (variable) restricted to the cell
key-value - a set of values, each associated with some key

Dimensional data are usually associated with aggregates. Key-value data are often taken from scraping the web. Even-history is used to describe events over some time period.
The nominal, ordinal, interval, ratio are not used to differentiate datasets. Rather, they are families of datatypes used to describe variables. Nominal data are those conforming to a finite set of categories with no other conditions (sex categories). Ordinal data are those conforming to an ordered finite set of categories, but the difference between adjacent categories is not necessarily uniform (Likert scale measures of satisfaction). Interval data are numeric with no zero (absence of quantity) defined (Celsius temperature). Ratio data are numeric with a defined zero (Kelvin temperature). These apply to any kind of statistical data.
The distinction between aggregate and unit data is based on the definition of the variables in the dataset. A dataset can contain both unit and aggregate data.
Access restrictions on data (e.g., public, restricted, private) are assigned by the business and can change over the life-cycle of the dataset.
The domain for a dataset is defined by the subject field that data apply to. However, some datasets are merged from others, so a merged set can have the combination of its constituents. There seems to be no restriction on the number of subject fields.
Mode of transmission is not definitional for a dataset, as a single dataset can be obtained multiple ways. The phases of GSBPM may not be useful, as a single dataset can pass through a phase without change. Further, the phases impose a usage criterion (data for collection; data for editing; etc.) that seems arbitrary and would be useless in another domain (outside statistics).
Similarly, the explorative, temporary, and organizational categorization is based on intent, rather than the data per se. Plus, the categorization could change without any change to the data. If we change the organizational structure described above (rectangular, etc.), then we should call that a new dataset.

Decided at the May 25 meeting:

create four sub-classes of coos:StatisticalDataset corresponding to the types listed by Dan above, say coos:RectangularDataset, coos:EventHistoryDataset, coos:DimensionalDataset and coos:KeyValueDataset.
all other categorizations should be rendered by properties with enumerated ranges (concept schemes)

Remaining questions:

should we create a Metadataset sub-class of coos:StatisticalDataset?
Where would graph data (RDF, property graph) go in the typology chosen?

Current GSIM model distinguishes between Data Set and Referential Metadata Set, both being sub-classes of Information Set. A metadataset sub-class of coos:StatisticalDataset might not be fully consistent with GSIM then.

Considering rdf data as triples, I think they would be best rendered by the long format (event history). The subject of the triple would be the identifier component, the predicate would be the variable descriptor component, the object would the variable value component.
If needed the named graph of the triple could be stored as an attribute component.
In the currently available documents for CDI-DDI reviews, classes are defined as WideDataSet and LongDataSet (see Part_2_DDI-CDI_Detailed_Model_PR_1.pdf) :
Wide Data: Traditional rectangular unit record data sets. Each record has a unit identifier and a set of measures for the same unit.
Long Data: Each record has a unit identifier and a set of measures but there may be multiple records for any given unit. The structure is used for many different data types, for example event data and spell data.
Multi-Dimensional Data: Data in which observations are identified using a set of dimensions. Examples are multi-dimensional cubes and time series. (Note that support is provided for time-series-specific constructs to support some legacy systems which are not based around the manipulation of multi-dimensional data “cubes”.)
Key-Value Data: A set of measures, each paired with an identifier, suited to describing No SQL and Big Data systems.
Do we agree to use those as definition for coos classes ?
I don't know how if the names of the classes will stay the same in the final specification of CDI-DDI or if they will be replaced by RectangularDataset, EventHistoryDataset. Either way we shouldn't we names coos classes the same way CDI-DDI does ?

Note that in the DDI-CDI model the actual name of the third type of data is Dimensional. We use Multi-Dimensional only informally in the documentation because people relate to that.
How are we introducing these concepts in the ontology? As data or as datasets?
In either case, I think the definitions need some work, they kind of look like explanatory text to me for the most part. Perhaps we could start all with something like "organized collection of data in which..." and then provide the characterization. That would be in line with the way they are all defined in GSIM and DDI-CDI.
That brings us to the question, I think, of what to do with the definitions when the classes already exists and are defined in one of the base models we are integrating. Do re-write them here or use them as-is from the source?

Florian, et al,
First, I’m happy using DDI-CDI class names if that is the wish of the group. I doubt the names of the already defined structure types will change.
I put a comment in my entry in the Doodle poll, and it got lost. I will try to repeat it here:
The problem I see with creating a new metadata set type is that you can’t distinguish a set of metadata by the set’s structure. The types I proposed are all based on structure. Whether data are metadata is determined by role (the idea metadata are used to describe), not data set structure. However, we may want to add to the list of allowable structures to include some other ways metadata are organized: unstructured (for text files) and hierarchical (for graphs, XML, RDF, and JSON).
I hope that helps.
As I tried to make clear in my comment in GitHub, there is no one way to organize a data set. One could organize an RDF graph in the long format. The model for a triple translates into the long format pretty well. However, you’d need an obvious way to state that the subject of one triple was the object of another. I’m not sure relying on a URI is sufficient, because URI’s don’t have to be unique.
As for the definitions, I want to see “and described” added after “identified” in the definition of Multi-Dimensional Data. Otherwise, yes. The definitions are fine.
The idea the dimensions identify a cell comes from SDMX (I think), and it is a seductive one. However, ultimately, I think it overloads what the dimensions are doing for each cell. Fundamentally, they describe the cell. An n-cube is based on a unit of analysis, or unit type, and the universe for each cell is that unit type specialized by the dimensions assigned to that cell. In more mathematical terms, it restricts the domain of the measure represented in the n-cube to that cell.
Those are two slightly different interpretations, but neither invokes identification for a cell. In particular, the terminological approach doesn’t depend on the existence of a measure. I contend that it is useful to define an n-cube without knowing a measure in advance. Because, one can use the same dimensions to stratify many measures.
Yours
Dan
From: Florian Vucko @.>
Sent: Thursday, May 27, 2021 5:12 AM
To: linked-statistics/COOS @.>
Cc: Gillman, Daniel - BLS @.>; Comment @.>
Subject: Re: [linked-statistics/COOS] Sub-classes of StatisticalDataset (#15)
CAUTION: This email originated from outside of BLS. DO NOT click links or open attachments unless you recognize the sender and know the content is safe. Please send suspicious emails to SECURE<EMAIL_ADDRESS>Considering rdf data as triples, I think they would be best rendered by the long format (event history). The subject of the triple would be the identifier component, the predicate would be the variable descriptor component, the object would the variable value component.
If needed the named graph of the triple could be stored as an attribute component.
In the currently available documents for CDI-DDI reviews, classes are defined as WideDataSet and LongDataSet (see Part_2_DDI-CDI_Detailed_Model_PR_1.pdf) :
Wide Data: Traditional rectangular unit record data sets. Each record has a unit identifier and a set of measures for the same unit.
Long Data: Each record has a unit identifier and a set of measures but there may be multiple records for any given unit. The structure is used for many different data types, for example event data and spell data.
Multi-Dimensional Data: Data in which observations are identified using a set of dimensions. Examples are multi-dimensional cubes and time series. (Note that support is provided for time-series-specific constructs to support some legacy systems which are not based around the manipulation of multi-dimensional data “cubes”.)
Key-Value Data: A set of measures, each paired with an identifier, suited to describing No SQL and Big Data systems.
Do we agree to use those as definition for coos classes ?
I don't know how if the names of the classes will stay the same in the final specification of CDI-DDI or if they will be replaced by RectangularDataset, EventHistoryDataset. Either way we shouldn't we names coos classes the same way CDI-DDI does ?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/linked-statistics/COOS/issues/15#issuecomment-849472732, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIVKL3AVKWLLMUL6IZ2TMDTPYEGDANCNFSM4HVB5EUA.

I apologize if this comment is out-of-time, but I have two questions after reading carefully all the comments:

I don't think that these four sub-classes of coos:StatisticalDataset corresponding to the types listed by Dan above, say coos:RectangularDataset, coos:EventHistoryDataset, coos:DimensionalDataset and coos:KeyValueDataset will be enough. What happens with other formats that can be found in the statistical domain like "Transposed", "Unstructured" or maybe "Blockchain" in the future. Is it possible to add sub-classes?
I think coos:StatisticalMetadataset should be a class at the same level than coos:StatisticalDataset

Edgardo,
I thought I made clear we can add to my original list of 4. But, transposed is probably similar to or maybe the same as event history. We’d have to dig into some examples to see.
As for StatisticalMetadataSet, I argued against that in a subsequent comment or email, specifically because there is no structure implied by simply being metadata. Further, I noted a couple of possible new structures related to metadata to add, and one was unstructured and the other graph or hierarchical.
I don’t understand “blockchain” as a structure. It is entirely possible I don’t understand blockchain well enough to know. Can you explain what it is about this that requires a new structure and what that looks like?
Yours,
Dan
From: Edgardo Greising @.>
Sent: Thursday, June 3, 2021 9:12 AM
To: linked-statistics/COOS @.>
Cc: Gillman, Daniel - BLS @.>; Comment @.>
Subject: Re: [linked-statistics/COOS] Sub-classes of StatisticalDataset (#15)
CAUTION: This email originated from outside of BLS. DO NOT click links or open attachments unless you recognize the sender and know the content is safe. Please send suspicious emails to SECURE<EMAIL_ADDRESS>I apologize if this comment is out-of-time, but I have two questions after reading carefully all the comments:

I don't think that these four sub-classes of coos:StatisticalDataset corresponding to the types listed by Dan above, say coos:RectangularDataset, coos:EventHistoryDataset, coos:DimensionalDataset and coos:KeyValueDataset will be enough. What happens with other formats that can be found in the statistical domain like "Transposed", "Unstructured" or maybe "Blockchain" in the future. Is it possible to add sub-classes?
I think coos:StatisticalMetadataset should be a class at the same level than coos:StatisticalDataset

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/linked-statistics/COOS/issues/15#issuecomment-853857563, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIVKL2PMXTARG2ZUM4TJMDTQ55QLANCNFSM4HVB5EUA.

Dan,
Thank you for your reply. Transposed is an old format that certain statistical datawarehouses used to implement to make it very efficient I/O by minimizing data transfer. The data is stored in multiple binary files, one for each variable with all the values in the same order. The nth value of each file compose a unit record. The structure can be complemented with a B-tree structure for indexing each unit. An example of a statistical product using such format is REDATAM.
I don't know enough about "blockchain", just that it is a back-linked list of "transactions" with a header containing metadata. I don't know if it will ever been used for statistical processes, but it could be. As far as I know, there are many ways of implementing it, using databases or even flat files, which makes it not different from the "Rectangular" or "Dimensional" types. In my understanding, it is not the data support format what differentiates the types, but the way the information is organized, and blockchain is different from a rectangular or dimensional dataset.
Regarding StatisticalMetadataSet, I don't fully understand your point on "there is no structure implied by simply being metadata".
If you use a DDI-C template for reference metadata, these metadata sets have a structure. Similarly, when you exchange reference metadata in SDMX there is always an MSD that defines the structure of the metadata set. And what is more important for me, is that these structures are different from and independent of the data structures. That's why I think that StatiticalMetadataSet is a different class.
Best,
Edgardo

Edgardo,
From your description of the transposed format, it is exactly what DDI-CDI means by event history data, and it is what I meant here.
Sounds like blockchain uses some format based on JSON, XML, RDF, or some other structure. We might not have to do anything. If there’s a blockchain standard that the industry has adopted with some new format, we might have to take a look.
Yes, the DDI’s (each member of the family) and SDMX specify formats for managing the metadata they define. But, those same structure types (hierarchy, graph?) could be used to organize data of many other kinds and applications. There is nothing inherent in the fact that the DDI’s and SDMX manage metadata that these structures exist. This is what I meant by “there is no structure implied by simply being metadata”. The fact some data are metadata does not, in itself, imply a structure.
Graph structures are typically used to organize metadata, and are used by the DDI’s and SDMX. But, with the metadata defined by the Dublin Core, the rectangular structure applies. Graph structures are increasingly being used to organize data in many domains, and rectangular structures have been used forever to organize some data. Therefore, a graph is not inherent to metadata.
If we define subtypes of a data set based on structure, a metadata set is not a structure type of its own.
Yours,
Dan
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/linked-statistics/COOS/issues/15#issuecomment-854497205, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIVKL7RDR6Z7ZWJH5YH6SLTRCIPNANCNFSM4HVB5EUA.

Another side to this discussion is to carefully look at the DDI-CDI and GSIM models to do a mapping. CDI Wide Data Structure is not the same as GSIM Unit Data Structure: the former is uniform, in the sense that each row has the same components/columns, whereas the latter is heterogenous, in the sense that each row might be associated to a different logical record. Just to keep in mind.

Ad hoc meeting was held on June 30th, Florian updated the ontology accordingly (see commit 217590e07e3944300bcfb141c01064c781266afc):

EventHistoryDataset renamed TransposedDataset
GraphDataset added (definition is missing)
no specific subtype for metadata: a property could be used
translations are missing
remaining question on mapping StatisticalDataset to GSIM (Information Set)

Regarding the property indicating that a dataset contains metadata, we could have a simple boolean "isMetadata" property or an object property like "metadataFor" whose domain could be the union of prov:Entity and prov:Activity (for process metadata). For metadata not attached to a particular process or entity (e.g. a statistical classification), the value of "metadataFor" could just be the "Official Statistics" individual.

Current state of things regarding the "products" domain:

Actually, limiting the range of the property might prevent some use cases (e.g. metadata on a prov:SoftwareAgent), so it is preferable to let the range open.


Miscellaneous minor corrections

A few cases of spelling errors - apparently mostly introduced by an
over-eager auto correct function.
Use sometimes rather than some times, when the meaning is "occasionally".
Consistently leave the introductory section out of the <gie>/</gie> block
(best practice - to allow any gie command given as example in the intro,
without gie interpreting it as an actual command).
Remove some superfluous whitespace, and a stray '#'.
Add some additional whitespace for consistency
Improve (?) the wording a few places

Thanks - I've added a small commit on top of this to correct a two minor things in the ellipsoid exercise.

Groovy - tendjewbellymud :-)


InvalidOperationException after upgrading to ASP.NET Core 3

On ASP.NET Core 2, our configuration looked like this:
services.AddSignalR().AddAzureSignalR(this.appSettings.SignalR.ConnectionString).AddJsonProtocol(options =>
{
    options.PayloadSerializerSettings.Converters.Add(new StringEnumConverter());
});

app.UseAzureSignalR(routes =>
{
    routes.MapHub<GeneralHub>("/generalhub");
});

After upgrading to ASP.NET Core 3, we changed our configuration according to this documentation.
Specifically:

For ASP.NET Core 3+, replace the above code with:
app.UseFileServer();
app.UseRouting();
app.UseAuthorization();

app.UseEndpoints(routes =>
{
    routes.MapHub<Chat>("/chat");
});

So we removed the call to UseAzureSignalR on IApplicationBuilder, but now we get the following error:
System.InvalidOperationException: 'AddAzureSignalR(...)' was called without a matching call to 'IApplicationBuilder.UseAzureSignalR(...)'.
Is there something missing in the documentation? Do we not need the call to AddAzureSignalR on IServiceCollection anymore? We need an ISignalRBuilder to be able to configure our PayloadSerializerSettings...

Which SDK version are you using? Try the latest one for .NET Core App 3.0
Only AddAzureSignaR is needed for ASP.NET Core 3+. Here is a sample code for it https://github.com/Azure/azure-signalr/blob/dev/samples/ChatSample/ChatSample/Startup.cs#L18

Turns out I wasn't using the latest version of the SDK. Thanks!


[PowerToys Run] Start PowerToys Run through the ActionRunner

Summary of the Pull Request
What is this about:
We have the fallback implementation for starting PT Run process if we can not delegate it to the explorer.exe.
We decided to remove ActionRunner as a middle process for starting PT Run and because it was made in the fallback this scenario hasn't been tested. Here we kind of revert it.
What is include in the PR:
How does someone test / validate:
It can be tested artificially by returning false from RunNonElevatedEx.
Quality Checklist

[X] Linked issue: #11955
[ ] Communication: I've discussed this with core contributors in the issue.
[ ] Tests: Added/updated and all pass
[ ] Installer: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Docs: Added/ updated
[ ] Binaries: Any new files are added to WXS / YML

[ ] No new binaries
[ ] YML for signing for new binaries
[ ] WXS for installer for new binaries

Contributor License Agreement (CLA)
A CLA must be signed. If not, go over here and sign the CLA.

The issue to investigate direct starting https://github.com/microsoft/PowerToys/issues/12047.

@davidegiacometti
It would be great if you can test the fix!

Sure! Let me build and test.

PT Run is starting as expected without errors on Windows startup.
Tested elevated and not.
There are no errors on event viewer related to PT Run.
[2021-06-30 21:00:16.7110] [INFO] [C:\Users\david\source\repos\PowerToys\src\modules\launcher\PowerLauncher\App.xaml.cs::46]
Starting PowerToys Run with PID=7272

[2021-06-30 21:00:16.7400] [INFO] [C:\Users\david\source\repos\PowerToys\src\modules\launcher\PowerLauncher\App.xaml.cs::51]
Runner pid=6116

[2021-06-30 21:00:17.3464] [INFO] [C:\Users\david\source\repos\PowerToys\src\modules\launcher\PowerLauncher\App.xaml.cs::92]
On Startup.

However in this branch ColorPicker doesnt work but don't think is related to this PR.
This is the error from event viewer.
Description: A .NET Core application failed.
Application: ColorPickerUI.exe
Path: C:\Program Files\PowerToys\modules\ColorPicker\ColorPickerUI.exe
Message: Error:
  An assembly specified in the application dependencies manifest (ColorPickerUI.deps.json) was not found:
    package: 'System.Deployment', version: '<IP_ADDRESS>'
    path: 'System.Deployment.dll'

Approving this since the issue seems related to https://github.com/microsoft/PowerToys/pull/11798 https://github.com/microsoft/PowerToys/blob/9c248cdcd22f2632de40a7cac0239d4ec1aa442d/src/settings-ui/Microsoft.PowerToys.Settings.UI.Library/Microsoft.PowerToys.Settings.UI.Library.csproj#L63


ARROW-7310: [Python] Expose HDFS implementation for pyarrow.fs

@ursabot crossbow submit test-conda-python-3.7-hdfs-2.9.2

depends on #6000

@ursabot crossbow submit test-conda-python-3.7-hdfs-2.9.2

@ursabot crossbow submit test-conda-python-3.7-hdfs-2.9.2

@pitrou @jorisvandenbossche it is ready to be merged


P2Ps should be allowed when ReferenceOutputAssembly=false even given TFM incompatibilities

From @AArnott on March 3, 2017 18:40
With VS2015 projects, I can have a P2P from a portable library to a net46 library by setting metadata on the project reference:
<ProjectReference Include="..\SomeNet46Lib\lib.csproj">
  <ReferenceOutputAssembly>false</ReferenceOutputAssembly>
</ProjectReference>

But with the .NET SDK projects, even with this metadata the build fails:

C:\Program Files (x86)\Microsoft Visual Studio\2017\d15rel\MSBuild\Sdks\Microsoft.NET.Sdk\build\Microsoft.NET.Sdk.Common.targets(73,5): error : Project 'C:\git\pinvoke\src\CodeGeneration\CodeGeneration.csproj'
targets '.NETFramework,Version=v4.6'. It cannot be referenced by a project that targets '.NETPortable,Version=v0.0,Profile=Profile92'. [C:\git\pinvoke\src\CodeGeneration\CodeGeneration.csproj]

This blocks scenarios where a P2P exists merely for the sake of ensuring build ordering but without the assembly reference. In my particular scenario, the referenced project provides a binary that the build of the portable library picks up for code generation purposes.
Copied from original issue: dotnet/sdk#939

Workaround: Try also adding SkipGetTargetFrameworkProperties=true metadata to the reference.

From @AArnott on March 7, 2017 5:58
Thanks @nguerrera. But that doesn't work either. That causes a referencing project A to build the referenced project B per A's TargetFramework value instead of B's TargetFramework.

Ah, I believe this would only happen if A is multi targeted. Is it?
Try adding UndefineProperties="TargetFramework" metadata as well.

Should we leave the issue active for making this scenario simpler, and/or work the way it used to?

Yes, this should work without the extra metadata.

From @AArnott on March 7, 2017 18:39
Yes, A is multi-targeted.
And that additional metadata did the trick. Thanks.
Should we leave the issue active for making this scenario simpler, and/or work the way it used to?

From @AArnott on April 11, 2017 13:31
This is badly broken. The workaround causes nuget restore to fail in VS (command line is fine) and also is related to a build failure that only occurs on some non-Windows machines including Travis CI Ubuntu.
I tried replacing this with a "project dependency" encoded in the solution file, and that fixed most of the symptoms, until I tried msbuild.exe my.sln when I learned that msbuild translates that solution dependency into a project reference during the build (@andygerlicher when did this feature get added?), with ReferenceOutputAssembly=false set (just as I wanted to do with my original ProjectReference item) and that of course repeats the original problem and the build fails because a net40 project can't depend on a netstandard1.5 project.
This inability to influence build ordering is really causing some pain here. Please fix soon!

From @AArnott on May 8, 2017 22:52
No. I finally gave up and checked the binary into git so I didn't need a project reference. I tried for days but never found a way that got dotnet build, msbuild, and VS to all work correctly at once. 😦

From @Sumo-MBryant on May 8, 2017 8:1
Has anyone found a workaround that is close to successful?
For a minimal netstandard1.X project reference to a netcoreapp1.X project:
SkipGetTargetFrameworkProperties fails in GenerateDepsFile (https://github.com/dotnet/sdk/issues/1020)
Interestingly enough when I restore and build from MSBuild directly the project.assets.json file is missing the project reference and builds successfully. When building from Visual Studio, the project.assets.json contains the reference with a broken framework "framework": "Unsupported,Version=v0.0" and fails to build.

From @rainersigwald on July 14, 2017 21:46
This can be worked around by adding an outside-the-norm order dependency in MSBuild, by way of a custom call to the MSBuild task.
<Target Name="WorkaroundSdk939" BeforeTargets="ResolveProjectReferences">
  <MSBuild Project="..\..\the\other.csproj" />
</Target>

Note that depending on your specific needs, you might need to be careful to preserve configuration and other normally-handled-for-you properties, or call a specific target.
@AArnott

msbuild translates that solution dependency into a project reference during the build (@AndyGerlicher when did this feature get added?)

This appears to have been added to MSBuild in the dev11 timeframe. @cdmihai went into detail on the process in https://github.com/Microsoft/msbuild/issues/2274#issuecomment-314850707. The current team doesn't know why it's necessary.

From @AArnott on July 14, 2017 22:52

Note that depending on your specific needs, you might need to be careful to preserve configuration and other normally-handled-for-you properties, or call a specific target.

Ya, that's what kills your proposed workaround in virtually all my scenarios. That would build the default configuration of the project, which could mean building all the target frameworks in debug mode, which is almost never what I would expect or need. Also, it would cause over-build, compiling twice etc. which can at least slow down the build, but also lead to symbols and DLLs not always matching up. It's a non-starter for me.
I've tried being very particular about passing in the right global properties to this call, but I guess there's a reason the ResolveProjectReferences target and its predecessors are so complicated. It's very hard to mimic.

From @rainersigwald on August 23, 2017 18:47
Workaround
Set
<AddSyntheticProjectReferencesForSolutionDependencies>false</AddSyntheticProjectReferencesForSolutionDependencies>

in the project that has the ProjectReference to the incompatible project. This prevents the elevation of solution build dependencies to ProjectReferences in AssignProjectConfiguration.
(Was poking around near this target for another reason and saw this.)

From @rainersigwald on August 7, 2017 13:41
@mhutch came up with an interesting workaround in https://github.com/Microsoft/msbuild/issues/2399#issuecomment-320470235:
<!-- workaround for https://github.com/Microsoft/msbuild/issues/2399 -->
<Target Name="WorkaroundMSBuildIssue2399" BeforeTargets="GetTargetFrameworkProperties">
  <PropertyGroup>
    <ReferringTargetFramework>$(TargetFramework)</ReferringTargetFramework>
  </PropertyGroup>
</Target>

In the referenced project.
That essentially disables the target-framework compatibility check for the referenced project, which could be somewhat dangerous (depending on the nature of other references to the project) but avoids this problem.

Moving to msbuild because after the double-evaluation fix, this compatibility check happens there in the context of the caller.

I would think SkipGetTargetFrameworkProperties=true would work now and that we should set this automatically when ReferenceOutputAssembly=false.

I just tried SkipGetTargetFrameworkProperties workaround and it works for me. Thanks!

Actually, I spoke too soon, it fails during pack:
C:\Program Files\dotnet\sdk\2.1.1-preview-007094\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targ
ets(167,5): error : Assets file 'C:\Users\rynowak\git\rynowak\Apparator\src\Apparator.Host\obj\project.assets.json' does
n't have a target for '.NETStandard,Version=v2.0'. Ensure that restore has run and that you have included 'netstandard2.
0' in the TargetFrameworks for your project. [C:\Users\rynowak\git\rynowak\Apparator\src\Apparator.Host\Apparator.Host.c
sproj]

my workaround
    <ProjectReference Include="..\ProjectToBuildFirst\ProjectToBuildFirst.csproj" PrivateAssets="All" Condition="$(TargetFramework)=='fake'"/>

@SimonCropp what does that accomplish? MSBuild would never build your project reference in that case. Is it that at least it works within the IDE that makes this workaround attractive?

FYI something in 15.5 seems to have broken my workaround while not resolving the original bug.
Fortunately setting  AddSyntheticProjectReferencesForSolutionDependencies  to True in the dependent project seems to work.

Oh, ok. @AndyGerlicher suggested a new metadata on ProjectReference telling the sdk what to do.
AutoTargetFrameworkOverride="true" where the sdks chooses from the returned tfm list, like pick the first one. Half joking, we could add to the p2p contract saying that projects should order the returned list from most preferred to least preferred :)
or
TargetFrameworkOverride="<user knows best tfm>"

Apparently TargetFrameworkOverride already exists as SetTargetFramework :)

It does, but the syntax is ugly SetTargetFramework="TargetFramework=net45". Ideally it would just be TargetFramework="net45", but I had a concern about that (potentially) tripping up the CPS project selector.

I'm running VS 15.5.5 and I cannot reference a 2.0 project in a UWP app.  Any workaround to this?  Thanks.

this is in milestone 15.6. but i can repro in <IP_ADDRESS>5
MSBuild auto-detection: using msbuild version '<IP_ADDRESS>5' from 'C:\Program Files (x86)\Microsoft Visual Studio\Preview\Community\MSBuild\15.0\bin'.
....
(_GetProjectReferenceTargetFrameworkProperties target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\Preview\Community\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(1603,5): error : Project 'C:\projects\costura\Costura.Fody\Costura.Fody.csproj' targets 'net46;netstandard2.0'. It cannot be referenced by a project that targets '.NETFramework,Version=v4.5.2'. [C:\projects\costura\Costura\Costura.csproj]
  C:\Program Files (x86)\Microsoft Visual Studio\Preview\Community\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(1603,5): error : Project 'C:\projects\costura\Costura.Tasks\Costura.Tasks.csproj' targets 'netstandard2.0;net46'. It cannot be referenced by a project that targets '.NETFramework,Version=v4.5.2'. [C:\projects\costura\Costura\Costura.csproj]
  C:\Program Files (x86)\Microsoft Visual Studio\Preview\Community\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(1603,5): error : Project 'C:\projects\costura\Costura.Template\Costura.Template.csproj' targets 'net46'. It cannot be referenced by a project that targets '.NETFramework,Version=v4.5.2'. [C:\projects\costura\Costura\Costura.csproj]
    0 Warning(s)
    3 Error(s)

@rainersigwald What's the plan for getting this fixed?
If I'm hitting this now, how am I supposed to work around it? I see https://github.com/Microsoft/msbuild/issues/2661#issuecomment-338808156, but how exactly am I supposed to be using that? Does that only work if I'm using solution-level dependencies, or is there a way to use a ProjectReference as well?
I'm asking because I've been able to get it working with a solution-level dependency, but because of other, unrelated problems, I can't use solution-level dependencies right now. Is there a way to use a ProjectReference and still have my solution build with MSBuild?

@rainersigwald any comments on the above question?

Also running into this. I'm trying to build a netcoreapp3.1 exe and include it in a NuGet package under /tools, e.g.:
<ItemGroup>
  <None Include="..\Faithlife.AssemblyEmbedder\bin\$(Configuration)\netcoreapp3.1\**" Pack="true" PackagePath="/tools/netcoreapp3.1/" />
</ItemGroup>

This works fine but ideally I'd like to have a reference so that Faithlife.AssemblyEmbedder is always built before my NuGet package csproj. I have a dependency in the sln file but dotnet pack ignores that.
Currently my workaround is to just do a separate dotnet build before the dotnet pack to ensure the project is built before it's included.


SQL Server resource update failed with error code AadOnlyAuthenticationIsEnabled

Bicep version
0.6.18
Describe the bug
When an Azure SQL Server has been created with Bicep template with azureADOnlyAuthentication set to true,  all subsequent run with the same bicep template for the existing SQL Server will failed with the following error:
code: AadOnlyAuthenticationIsEnabled
message: Azure Active Directory Only Authentication is enabled. Please contact your system administrator.
To Reproduce
1- Use Bicep to create an Azure SQL Server with azureADOnlyAuthentication set to true
2 - Run the same Bicep template the second time and it will failed with error code AadOnlyAuthenticationIsEnabled
Additional context
The Bicep for SQL Server works if the existing SQL Server resource does not have azureADOnlyAuthentication set to true

Here is the response I got from the SQL team:

Changing the password of a SQL server is not permitted if AAD Only Auth is enabled. If the customer uses bicep to deploy a server, the bicep template might contain the password during creation and also enable aad only at the same time. Subsequent calls to upsert the password/server hence fail with said error.

I asked what would happen if the password did not change and got this back:

We can’t check if the password is different as that would reveal information to a potential attacker. If supplying the password works when the password is the same (or returns a different error than when it is different), this adds another vector for brute forcing the password. Instead, we can only check if the password is specified. If it is specified and AAD-only is enabled, the operation will fail.

I am not sure about the validity of this statement. My understanding is other resource types like VM and AKS allow a repeat PUT with a password.

I think what they are saying translates to:

changing the password of a SQL server admin login is not permitted when AAD-Only Auth is enabled
deployment cannot return a separate result (success v failure) depending on whether the password is the same, because that would be an attack vector
therefore it is impossible to re-deploy a configuration that includes the adminPassword with azureADOnlyAuthentication set to true even if the password wouldn't be changed
adminPassword is required for creation of SQL Servers
with all of the above factored in, there is no way to deploy adminPassword with azureADOnlyAuthentication set to true twice with the same configuration

The key here is that adminPassword is required for creation of SQL Servers. Fortunately, it's not required for updates. Therefore, a solution to the above is to remove administratorLogin and administratorLoginPassword from subsequent updates. It doesn't matter if these values change in your environment, because they cannot be changed once AzAD-only auth is enabled.
@description('If this is the initial creation, we need to pass sql login information. Otherwise, we cannot.')
param createSqlServer bool = false

resource sqlServer 'Microsoft.Sql/servers@2021-08-01-preview' = {
  properties: {
    administratorLogin: createSqlServer ? sqlServerAdministratorLogin : null
    administratorLoginPassword: createSqlServer ? sqlServerAdministratorPassword : null
    // ...
    administrators: {
      administratorType: 'ActiveDirectory'
      // ...
      azureADOnlyAuthentication: true
    }
  }
}

If your SQL Server has azureADOnlyAuthentication disabled, the above template won't work. You'll have to enable it via Azure portal or az cli. See the issue linked at the end of this comment.
If you have an existing SQL Server with AzAD-only auth enabled, the above will re-deploy.
I haven't tested whether it will work on the initial deployment because I only ran into this issue after the initial deployment.

That still leaves the following issue:

Cannot re-enable AAD-Only Auth via bicep when an existing sql server has it disabled: https://github.com/Azure/bicep-types-az/issues/1436#issuecomment-1478124394

Here is the response I got from the SQL team:

Changing the password of a SQL server is not permitted if AAD Only Auth is enabled. If the customer uses bicep to deploy a server, the bicep template might contain the password during creation and also enable aad only at the same time. Subsequent calls to upsert the password/server hence fail with said error.

I asked what would happen if the password did not change and got this back:

We can’t check if the password is different as that would reveal information to a potential attacker. If supplying the password works when the password is the same (or returns a different error than when it is different), this adds another vector for brute forcing the password. Instead, we can only check if the password is specified. If it is specified and AAD-only is enabled, the operation will fail.

I am not sure about the validity of this statement. My understanding is other resource types like VM and AKS allow a repeat PUT with a password. If anyone can provide evidence of that, it might move this along, otherwise, I am not sure what options there are.

Thanks for checking with them. The explanation why the password should not be changed during the update makes sense, but their implementation of IaC doesn't. The most convenient way for us (Azure users) would be for the RP team to solve the problem with a conditional logic in their RP implementation that just disregards any provided logins/passwords during the update of 'AAD only' sql servers.
Does anyone know what's their GH repository to create this issue for them?

I think the ideal situation would be if the admin user was completely removed when AadOnly is enabled. When one disables the flag, they would be forced to create a user at the same time (at once with bicep, or manually in the portal or CLI). I imagine SQL Server requires at least one internal user though since one is required to create the instance even when AadOnly is immediately enabled. Alternatively, they could accept updates to the user even if AadOnly is enabled since the user is still there in the table.
I don't think it makes sense to ignore the user and pass configured in ARM. The user and password does still exist, so if one disables AadOnly via the portal or Az CLI after rotating keys and re-deploying, this user and password would not have the last values passed to it if SQL Server simply ignored it during deployment. In the event an organization rotated their keys specifically due to a compromised secret, that secret would now be active again, defying the expectations of infrastructure as code. Instead, an error is returned, and I think it makes sense for deployers to change the configuration not to pass those values with related documentation as to why.
In reference to your question, I don't think Microsoft internal teams have Github repositories. Open source projects and documentation teams have various levels of monitoring and sometimes Microsoft team members will pass info from open source channels through back channels for confirmed issues, but Azure SQL / SQL Server internal teams for instance do not have a Github repository. Direct communication is generally done through Microsoft support or Azure Technical Support which I believe are both paid features, as described here: https://learn.microsoft.com/en-us/sql/sql-server/sql-server-get-help?view=sql-server-ver16

I get the same error with ARM template as well.
When AAD Only Auth is enabled
{"code":"AadOnlyAuthenticationIsEnabled","message":"Azure Active Directory Only Authentication is enabled. Please contact your system administrator."}
When AAD Only Auth is not enabled
{"code":"InvalidParameterValue","message":"Invalid value given for parameter AzureADOnlyAuthentication. Specify a valid parameter value."}

Boo, I just ran into this issue wondering why I even have to use a password. 😭  Everything is passwordless these days.  Indeed, not a single password is set/used in nearly 30 other components that I am deploying to Azure via Bicep.  Talk about an inconvenient relic.

This has been broken for years now.  It should not be that hard to set the resource manager to ignore an attempt to update the SQL admin password if the resource already exists and azureADOnlyAuthentication is set to true.

Seeing this issue as well.
One challenge we're facing with the workaround, and thanks for the detailed comment and example implementation @Nirnette, is that we have separate Azure Policies around storage account security settings, which independently blocked use of the Deployment Pipeline activity which creates a Storage Account, then deletes that same Storage Account.
Your own security policies may or may not block this Bicep-generated Storage account, just wanted to call out yet another complexity spurred by this MSFT-side bug.

Hello @awbarbeau ,
As of today there is no other way to validate properly a resource exists or not, but I heard something may be coming using "existing", but I don't know more.
I'd recommend adding an exception on the pipeline while it deploys, but we run publicNetworkAccess (no anonymous users) on Storage accounts on Deny, and my work around works, so it must be another one blocking?

@Nirnette, it ended up being the combination of rules around the short lived container that the deployment script option requires that halted us. Again, not a problem in the solution, just a case where security policies can impact feasibility.
Separately, making some headway on a case with relevant product teams for Bicep and Azure SQL. I'll push for a reasonable fix!

I encountered the same problem.  Here is a way to avoid the error:

remove the administratorLogin and administratorLoginPassword parameters from your code. Since you decided to use the azure AD-only authentication, the admin user is not a necessity.
Actually, without admin user and password parameters set, Azure will create a random admin user on your behalf, something like "cloudxxx"
now repeatedly executing the bicep code will not report any errors.
However, if you have already set the admin user and password in your first deployment, it will not work. You have to remove the admin user and password from the Azure portal and the two parameters from your code.

Not only, does updating not work. Also re-deploying with the same config does not work.
In my case, I started with a template deployment of our DEV stage:
var stage = 'DEV'

resource sqlServer 'Microsoft.Sql/servers@2023-08-01-preview' = {
  name: '${serverName}-${stage}'
  location: constants.location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    version: '12.0'
    minimalTlsVersion: '1.2'
    publicNetworkAccess: 'Disabled'
    administratorLoginPassword: 'some-trival-pw-of-course'
    administrators: {
      tenantId: constants.tenantId
      azureADOnlyAuthentication: false
      administratorType: 'ActiveDirectory'
      principalType: 'Group'
      login: constants.securityGroups.sqlAdmin.name
      sid: constants.securityGroups.sqlAdmin.id
    }
  }
}

This generated a local sql admin named CloudS...... Also a AD security group was assigned admin access.
Then with a different stage, I tried AAD only auth by editing the Bicep template. Note var stage = TEST and azureADOnlyAuthentication: true and ommitting administratorLoginPassword.
var stage = 'TEST'

resource sqlServer 'Microsoft.Sql/servers@2023-08-01-preview' = {
  name: '${serverName}-${stage}'
  location: constants.location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    version: '12.0'
    minimalTlsVersion: '1.2'
    publicNetworkAccess: 'Disabled'
    administrators: {
      tenantId: constants.tenantId
      azureADOnlyAuthentication: true
      administratorType: 'ActiveDirectory'
      principalType: 'Group'
      login: constants.securityGroups.sqlAdmin.name
      sid: constants.securityGroups.sqlAdmin.id
    }
  }
}

This deployed and worked really well.
BUT now when I go back to re-deploying the DEV stage, it also does not work when deploying the orignal template
var stage = 'DEV'

resource sqlServer 'Microsoft.Sql/servers@2023-08-01-preview' = {
  name: '${serverName}-${stage}'
  location: constants.location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    version: '12.0'
    minimalTlsVersion: '1.2'
    publicNetworkAccess: 'Disabled'
    administratorLoginPassword: 'some-trival-pw-of-course'
    administrators: {
      tenantId: constants.tenantId
      azureADOnlyAuthentication: false
      administratorType: 'ActiveDirectory'
      principalType: 'Group'
      login: constants.securityGroups.sqlAdmin.name
      sid: constants.securityGroups.sqlAdmin.id
    }
  }
}

Also this does not work (dropped administratorLoginPassword):
var stage = 'DEV'

resource sqlServer 'Microsoft.Sql/servers@2023-08-01-preview' = {
  name: '${serverName}-${stage}'
  location: constants.location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    version: '12.0'
    minimalTlsVersion: '1.2'
    publicNetworkAccess: 'Disabled'
    administrators: {
      tenantId: constants.tenantId
      azureADOnlyAuthentication: false
      administratorType: 'ActiveDirectory'
      principalType: 'Group'
      login: constants.securityGroups.sqlAdmin.name
      sid: constants.securityGroups.sqlAdmin.id
    }
  }
}

And this does not work either (dropped administratorLoginPassword and administrators):
var stage = 'DEV'

resource sqlServer 'Microsoft.Sql/servers@2023-08-01-preview' = {
  name: '${serverName}-${stage}'
  location: constants.location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    version: '12.0'
    minimalTlsVersion: '1.2'
    publicNetworkAccess: 'Disabled'
  }
}

Workaround is to run once Disable-AzSqlServerActiveDirectoryOnlyAuthentication or Enable-AzSqlServerActiveDirectoryOnlyAuthentication before deploying the Bicep/ARM template. This should set the property properties.administrators.zureADOnlyAuthentication to the desired value which could be missing if you have previously used the child resources to set Entra authentication. Unfortunately poor execution of SQL team where separate resources were introduced for setting Microsoft Entra authentication initially and later they have incorporated these settings in the SQL server resource proprieties in order to be able to create SQL server with Entra authentication from the start without havign to specify login account. That is my implementation of the issue without having internal information.


No way to set jsonData.prometheusType or jsonData.prometheusVersion for datasource

This might be a feature request not a bug, I apologize if I did it wrong. I will be submitting a PR shortly that fixes this.
Describe the bug
It appears that there is no way to set jsonData. prometheusType and jsonData. prometheusVersion on datasources. These need to be set to get Grafana to work well with Google Managed Prometheus. If you don't set them, metric name auto completion lookup doesn't really work.
Version
v4.10.0
To Reproduce
Attempt to apply CRD with jsonData. prometheusType or jsonData. prometheusVersion set.
Expected behavior
A way to configure jsonData. prometheusVersion and jsonData. prometheusVersion.

I finally found customJsonData which I assume will be the suggested solution.


Add deployed site link to readme

Resolves #63 and #89 - how do you use this tool without installing it on your laptop

How is this related to #89?


Use current and stable chrome version in Concourse pipeline

We encountered a problem with the business-volunteers form pipeline, in which tests were failing due to capybara needing headless Chrome (https://github.com/alphagov/govuk-coronavirus-business-volunteer-form/commit/27d5027cff0332a172628fa46465ab96fd40a55d).

Updating our Concourse Dockerfile here as a precaution and to keep consistent with the business-volunteers form.

How to review
:ship: Since this service is continuously deployed, all testing must be done
before the pull request is merged. :ship:
Describe the steps required to review and test the changes.
Links
Trello

Codecov Report

Merging #420 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #420   +/-   ##
=======================================
  Coverage   99.28%   99.28%
=======================================
  Files          71       71
  Lines        1955     1955
=======================================
  Hits         1941     1941
  Misses         14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55ff0a9...de45255. Read the comment docs.


Docker image 14.0.1 is 14.4.1 actually?

I am asking this because I still want to use 14.4.1 and not yet the latest 15.0.0

It seems that the 14.0.1 is somehow a bit strange.
I can not really say which version it is, but between the 14.3.0 and the 14.4.1 there are really not many changes.
@adrian-evo
May i ask why you do not want to go to 15.0.0?

@Snooz82 I was testing 14.4.1 today and planned to deploy also docker image. I was not expecting 15.0.0 and I can see some backward incompatible changes which I didn't checked yet.

We found a bug in our release process and we need to fix the problem.

@allcontributors please add @adrian-evo for bugs

This is actually fixed now


Search and Sort not working

A few days ago table search and sorting was working just fine. but now suddenly it stopped to work. IDK why its happening. I tried to reinstall it but still nothing, kindly help.
Versions using,
"react-bootstrap-table-next": "^1.4.2",
"react-bootstrap-table2-toolkit": "^1.1.1"

Hello @amad-aleem123 you information is not enough for me to reproduce your issue, Itry to reproduce but I can not make it. please write more information and give me a simple or minimal example here or on the codesandbox.io.
I will reopen this issue until you know how to give people enough information, thanks.

I don't have anything else to tell. I just know everything was working absolutely fine, then suddenly it stopped working. Kindly tell me what more information you want so I can elaborate.

If you still don't understand my problem, then let me share my screen with you on Skype.


Add Verk.RetrySet.delete_job/1

It will accept the job with the original json or the json itself

:+1:


composite.rotate cannot rotate composite around the  point

Looking at this string of wooden wheels, I created them with a for loop for observation. I used "Composite.rotate" to try to make it rotate around a point, but it was always unsuccessful.

No matter how I change the "point", it won't have much impact unless I change the number greater than 100
Composite.rotate(bd, 1, {x:1,y:5}, true)
By the way, how should "rotation" be used? Is it the angle of rotation?


Add support for DejaVu Sans Mono for Powerline font

I was using DejaVu Sans Mono for Powerline font when I was using clink 0.4.9 and some Powerline script for it.
I've switched to your clink version and clink-flex-prompt and some of the special characters aren't rendered correctly.
The battery % seems to work sometimes ?

And one of the icons for git branches seems to be off as well

It's not a big issue for me at the moment because I've since switched to MesloLGS NF which has all the icons in all the correct places and it's working great for me so far :)

You can override icons and symbols. Refer to stuff near the top of flexprompt.lua.
Maybe tomorrow I can share an example, if it isn't making sense from reading flexprompt.lua.
I'm not going to "support font X (and Y, and Z, and ...)".
It's flexible and configurable, on purpose.  There are multiple different symbols someone might want to use for "battery is charging" or for "git icon" or "folder icon" and so on. Choose the symbols you want to use, and set the variables. You don't need to modify the flexprompt scripts -- you can set them in a script of your own.

That's a sensible and sane approach :)
Thanks for info, I'll have a look a bit later.
But right now I'm really liking the MesloLGS NF font so I'm in no hurry to hunt for font codepoints in various fonts

The branch symbol is ok on DejaVu Sans Mono for Powerline, but the symbol used for the purple color, that's the wrong one (I think for branches that are not available yet on the remote).
Here's the config as seen with the DejaVu Sans Mono for Powerline font:

Here's the same part of the config as seen with MesloLGS NF font:

Here's the command prompt with MesloLGS NF font:

And here's the same command prompt with DejaVu Sans Mono for Powerline font:

The charging and unpublished icons are missing.
I also have another DejaVu font installed, "DejaVuSansMonoForPowerline NF":
Here's the command prompt with that font:

So it looks as though "DejaVu Sans Mono for Powerline" is missing the charging and unpublished icons, but "DejaVuSansMonoForPowerline NF" has those icons too.


Amazon error: Need to first confirm Terms and Conditions

My Amazon setup is new, so I received this error when running provision.sh:
TASK [cloud-resources : AWS Create All in one OpenShift Node] ******************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Instance creation failed => OptInRequired: In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit https://aws.amazon.com/marketplace/pp?sku=aw0evgkw8e5c1q413zgy5pjce"}

Need to approve the billing rate of 1.2 cents per hour for CentOS.

Also; AFTER accepting said billing, you need to wait a bit or you'll get the following error:
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Instance creation failed => PendingVerification: Your account is currently being verified. Verification normally takes less than 2 hours. Until your account is verified, you may not be able to launch additional instances or create additional volumes. If you are still receiving this message after more than 2 hours, please let us know by writing to<EMAIL_ADDRESS>We appreciate your patience."}

This should be solved with #14


Task: Remove the BIN role from the diagrams

What is the task about?
After all other issues from the components are closed:

https://github.com/vmware/repository-service-tuf-api/issues/224
https://github.com/vmware/repository-service-tuf-worker/issues/165
https://github.com/vmware/repository-service-tuf-cli/issues/174

then we can remove the BIN role from the diagrams as well.
Parent feature

https://github.com/vmware/repository-service-tuf/issues/28

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I tried removing all instances of the bin word from the repository.
It seems that the only place where we are using it is inside docs/diagrams/1_1_rstuf_metadata.puml file.
There I remove the delegation of targets to bin and remove all places where bin delegates to any of the other bins.
Still, after I run make docs successfully I see that I have not removed bin from this diagram:

I guess it's because, at the moment of writing this, the other components haven't yet removed bin in their main branches.
Is this diagram from another component?

After we update the submodules I can try and retry building the diagrams.
@kairoaraujo did you find another way of building the diagrams or should I try the same way as we did up to now?

@MVrachev We can do it in the same way.
Let's fix the FT (https://github.com/vmware/repository-service-tuf-api/pull/239), and we are good to go here.

Closed by #232


Simplier intervals

@for $i from 1 to 3 {}

reproduces only 1 and 2, becaus you copy JS for statement logic? Like for ( i = 1; i < 3; i++ )?
I think we should not copy bad design from JS. @for $i from 1 to 3 should be more human and reproduses 1, 2 and 3.
It will also remove through keyword. And one keyword is always better.
/cc @antyakushev

@ai this copies sass syntax. So the change can be quite unexpected for the sass users, though I agree that it does look better.

Right now there is no plugins, that copies Sass syntax 1-to-1.
So I think it is a good place to fix this bad design.

Ok, will do :smile:


Refresh jobs list on Dashboard

Some people requested for automatic refreshing the jobs page whenever any change has been applied in the meantime or new job popped in.
We can use Socket.io library (which is already used by racetrack-client and Lifecycle for following logs), it works on top of websockets, falling back to long polling.

I agree websockets are a good choice of mechanism here.
Only downside is, websockets assume the client is a web browser. So it makes it harder to share serverside endpoints between browser and CLI clients.

Another downside (or challenge) is the fact that we got 3 replicas of a Lifecycle server. And when some change has been applied to one of them, it has to be broadcasted further to notify all of the connected clients. And that would need some sort of MQ or cache, like Redis (or shared filesystem volumes?).

Huh why? All 3 replicas would have their own websocket connection to the backend.

Huh why? All 3 replicas would have their own websocket connection to the backend.

"All 3 replicas" are "backend". To be more specific, I wonder how to notify "Lifecycle 1" server, which one client is connected to, whereas some other guy submitted his change to "Lifecycle 3".

I found this: https://tsh.io/blog/how-to-scale-websocket/. I think the Horizontal scaling with WebSocket Issue #2: Broadcasting section is the problem that you described.
It's in JS (Node.js) but the overall idea behind the solution is more important.


SP5: WPF control vesigner view looks bad

It would be great if we could display something more useful instead of this...

@swharden It would be hard to change, because it is the way SKElement was designed in SkiaSharp.
I found an opened issue at SkiaSharp #2110, to allow rendering during design, but it was not implemented in the source code.

Thanks for your research ingo this @KroMignon!


Check for presence of controller element in serverMessage

Type of PR (feature, enhancement, bug fix, etc.)
Bug fix
Description
Fixes #459
Why should this be added
Avoid unnecessary JS exceptions.
Checklist

[x] My code follows the style guidelines of this project
[x] Checks (StandardRB & Prettier-Standard) are passing
[x] This is not a documentation update

I slightly restructured your commit so that we could eliminate a redundant comparison.


[🐞] code used only by a loader$ gets loaded in the browser

Which component is affected?
Qwik City (routing)
Describe the bug
If a file exports 2 functions, and one is used only by a loader$ and the other is used only by a component, during SPA navigation, the entire file is loaded into the browser.
This will cause an error if the file also uses server only code or imports a server package.
It also could potentially leak sensitive code.
The issue occurs during development with pnpm start, Im also experiencing it on my preview server.
This is using the latest code from qwik-build and qwik-city-build
Reproduction
See below
Steps to reproduce
Create a file that has 2 pieces of logic:
// Some backend code
export const authenticateUser = async (): Promise<string> => {
  console.log("This should not run in the browser");

  return "my-data";
};

// A hook that can run on both environments
export function useAuth(userData: string) {
  const user = useSignal<{ user: string }>({ user: userData });

  useContextProvider(USER_CONTEXT, user.value);
}

Then in a child route, use both pieces of logic:
export const childLoader = loader$(authenticateUser);

export default component$(() => {
  const { value } = childLoader.use();
  useAuth(value);

  return ...
});

Finally, in your browser, navigate to the child page using a Link and you will be able to search for "This should not run in the browser" in your browser.
System Info
System:
    OS: macOS 12.0.1
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 23.43 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 18.12.1 - ~/.nvm/versions/node/v18.12.1/bin/node
    Yarn: 1.22.18 - /usr/local/bin/yarn
    npm: 7.13.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 109.0.5414.119
    Firefox: 109.0
    Safari: 15.1
  npmPackages:
    @builder.io/qwik: BuilderIO/qwik-build#main => 0.16.2
    @builder.io/qwik-city: BuilderIO/qwik-city-build#main => 0.1.0-beta9
    undici: 5.16.0 => 5.16.0
    vite: 4.0.4 => 4.0.4

Additional Information
No response

can you prepare a small repo case? during prod build it should be treeshaken!
The issue during npm start is expected, we don't do treeshaking during development.
Inlining the logic in loader$() will prevent many of this issues:
export const childLoader = loader$(async (): Promise<string> => {
  console.log("This should not run in the browser");

  return "my-data";
});

Ok try this with a preview build!
You should see one chunk is like 1.5mbs and when you navigate twice with the buttons theres an error in the console which will point you to a loader$s package being imported.
https://github.com/DustinJSilk/qwik-loader-treeshaking
I've inlined my loaders now so its not a high priority for me particularly

I cant reproduce the issue in the repo you provided, the code does not leak to the browser in preview mode

Thats odd.. Would you mind taking a look at this recording where the issue exists?
https://drive.google.com/file/d/1Y0JQGeiWV0WWCUaexrJ3QgpbzfgyNiRI/view

Hello. I worked with those kinds of issues before. I think changing loader$ to this should help:
export const childLoader = loader$(() => {
  authenticateUser();
});

After this change auth.ts is not fetched on the client.
I assume this is related to the compiler analysis of AST. By making this arrow function -> splitting server/client code from server-only is easier. I hope this is the correct way of thinking about this.

I'm experiencing this same issue in my code base as well. I have had to wrap some of my server code in server$ to prevent it from leaking to the client.  I tried to reproduce on StackBlitz but was unsuccessful.
I tested using a routerAction$, routeLoader$, and useVisibleTask$ all in the same component which I suspect is the issue in my code base, but it works fine on StackBlitz.  If I eliminate useVisibleTask$ in my component the issue goes away, so it definitely seems to be a relationship between the loader and the useVisibleTask$ call.

This issue should be fixed in main! next release should have the fix!


APIView - Reviews dropdown should show more metadata to differentiate reviews

New dropdown in review page shows all reviews for a given package name (this is based on the package name of current review). Dropdown lists many reviews with same name for a package and it makes is difficult to identity what needs to be verified. We need to update dropdown to show more differentiating properties to show which one is pull request, manual and automatic in addition to any label available in review metadata.

Resolved in https://github.com/Azure/azure-sdk-tools/pull/2785


kits-schema.json not found when opening cmake-kits.json file

Brief Issue Summary

Open a cmake-kits.json file.

Notice the warning from the screenshot.

CMake Tools Diagnostics
No response
Debug Log
No response
Additional Information
No response

@DragosMiloiuNXP Thanks for notifying us! I'll place this on the backlog to be investigated and mark it as a bug. Thanks.


Bazel + CI

We should run our Bazel build/test/coverage in CI.
There's no way to know if a PR breaks the Bazel support other than manually running it.

I can run the lint-test-build task using Bazel on CI. However, the builds take much longer than with cargo (build|test).
See https://app.circleci.com/pipelines/github/liftedinit/many-rs/801/workflows/abfa41cb-682d-48c3-ae8b-377c23ddb98a/jobs/1972


[FEATURE REQ] Investigate way to handle additional parameters in model constructors.

This is follow up from https://github.com/Azure/azure-sdk-for-java/pull/9283 .
Every time server side returns new piece of data we're adding properties to models. That results in necessity of adding new constructor(s) that capture new parameters and leaving old constructors in place for backwards compatibility. We should figure out a way how can we avoid multiplying constructors.

I think this can be resolved by exposing an interface with getters on the public API and hiding the actual implementation of the type in the implementation package. See how blob changefeed exposes BlobChangefeedEvent and BlobChangefeedEventData

Yeah, I think this is the answer, hide implementation behind interface.

Closing this as resolved as there are patterns in place for evolving constructors to use the implementation interface pattern to allow the number of class properties to grow without needing to add new constructors each time.


chore: Release v1.9.28

This is a promotion branch for the release v1.9.28

/build-deploy-preview skip-tests=true recreate=true

/ok-to-test

/ok-to-test


docs/stdenv: Document updateScript features

Description of changes
This was removed in https://github.com/NixOS/nixpkgs/commit/c1b05442ffd6cf3cf529cad469bebe8169b156e9 for stabilization but it has worked quite well.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/tweag-nix-dev-update-40/23480/1

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/2023-03-21-documentation-team-meeting-notes-34/26619/1


OSM Lat/Lon precision requires more than f32

Hi! Firstly, thanks for the crate!
Currently in osmio, Lat/Lon is represented by a f32, but sometimes this is not precise enough for osm data, which is spec'd to 7 decimal places.
from https://wiki.openstreetmap.org/wiki/Node:

decimal number [...] with 7 decimal places
...
Do not use IEEE 32-bit floating point data type since it is limited to about 5 decimal places for the highest longitude.

My understanding is that the internal representation is a scaled 32bit integer

A 32-bit method used by the Rails port is to use an integer (by multiplying each coordinate in degrees by 1E7 and rounding it

So, as I understand it, 1.234567890 degrees would be stored as 12345678i32
Looks like libosmium also uses this representation internally.
My intuition is that the lon/lat types you've currently defined should become structs backed by an i32, with some in/out methods, including something like lat.to_f64() (but probably not a lat.to_f32()).
And, just to be clear, I expect that this would break a lot of consumers code, who are already coding around these things being f32's. For a lot of end user applications, the precision probably doesn't make a difference, but I think for a parsing library like this (and libosmium) it makes a lot of sense to respect the upstream precision model.

The 7 rounded decimal places for coordinates in degrees define the worst error of longitude to a maximum of ±5.56595 millimeters on the Earth equator, i.e. it allows building maps with centimetric precision. With only 5 decimal places, the precision of map data would be only metric [(within a meter)], causing severe changes of shapes for important objects like buildings, or many zigzags or angular artefacts on roads.

What do you think @rory?
If you're interested in the concept, but don't have time for the work, this is something I'd be interested in working on.

Oh I should add that an alternative would be to use f64 instead of f32. I think this could solve our precision problem, and probably be easier for downstream users to adapt to, but it'd takes twice as much space as necessary.
That's why I'd advocate for having the internal storage be an i32, and and a getter for f64 representation for those that want to do floating point with it.

That's a good idea. If you want, you could work on a PR

Ok, will do!


"Ghioncani" is listed in the drop-down menu

Description:
"Ghioncani" is listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Ghioncani" in the search bar.

Expected results
The location is listed in the drop-down menu as "Ghioncani, Alba (Întregalde)".


Comments not reaching the bottom of the UI

The comment box does not reach the bottom of the window. Instead, there is some empty space between the box and the bottom of the UI.

Expected behaviour: The comments box reaches the bottom of the screen, like the other text boxes:

#2357


Question about wifiduck/hardware/connections.md

I have a question regarding the information on the Wiki at wifiduck/hardware/connections.md

Would it be possible to change the pins used on the Arduino?
Currently the Pins on the Arduino (ignoring RAW and GND) are;
Pin 3 on the Arduino connecting to D1 on the ESP8266
Pin 2 on the Arduino connecting to D2 on the ESP8266
Could this be amended to;
Pin 14 on the Arduino connecting to D1 on the ESP8266
Pin 15 on the Arduino connecting to D2 on the ESP8266
(Essentially any number pin on the same side as the RAW and GND pins)
Just thinking it might make it easier to make a smaller end result with minimal wiring.
(Very new to Arduino etc, so apologies if this is just straight up a dumb question. Thanks)

The WiFi Duck project is using I2C pins (SDA,SCL) and these (as far as I know) can not be changed software wise on the Atmega32u4. So no you can't switch them to pin 14 and 15 :(

The WiFi Duck project is using I2C pins (SDA,SCL) and these (as far as I know) can not be changed software wise on the Atmega32u4. So no you can't switch them to pin 14 and 15 :(

I have the dstike  wifi duck all in one board.  Not sure what the name of the board would be.  I had the board less than a day and I got the internal connection problem.  How could I fix that issue?
Thanks


roll back from 3.0 to 2.x

Ionic version:  (check one with "x")
[ ] 1.x
[ ] 2.x
[x] 3.0
I'm submitting a ...  (check one with "x")
[ ] bug report
[ ] feature request
[x] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
How can i roll back to 2.x?
i've updated ionic and package.json :(
I can choose the version of the doc on your website in API/Component part
but NOT in Native part
I cant find the 2.x version of NATIVE :( :( :(

[x] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/

Also, Native 3.x works with Ionic 2

Hello! Thanks for opening an issue with us! As this seems like more of a support question i will urge that you ask this question on our forum or on our slack channel. Thanks for using Ionic!


PP-1857 [REVIEW-ONLY] Upgrade Notify client to version 3.1.1-RELEASE

We are running an old version of Notify client, this upgrade will allow us to send emails in adminusers therefore we can implement Forgotten Password and Invite email notifications.

There were errors, for info, please see...


feature make WaitForLocalExchangeBuffer timer merge

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


[http] Link between Connection layer and Request handler

Hi,
I'm using this kind of code to handle connection timeouts
$loop = React\EventLoop\Factory::create();

$socket = new React\Socket\Server($loop);
$socket->on('connection', function ($conn) use ($loop) {
    $func = function () use ($conn) {$conn->close();};
    $timer = $loop->addTimer(1, $func);
    $conn->on('data', function ($data) use ($loop, &$timer, $func) {
       $timer->cancel();
       $timer = $loop->addTimer(1, $func);
    });
});

It's working OK. But for some GET request with long results (search) I've to cancel the timer from the request handler (because it  timeouts).
But I don't know how to connect the two worlds...

We do, what you're looking for are connection and request timeouts. We have this feature planned for a future release.

Nice to know.
That said, it would be nice to be able to pass data along connection to request.
Connection could have attributes. And connection could be injected in the request. Or exposing the 'headers' event that knows the 2 objects.

What is your use case for that? You have indirect access to all properties on the connection in the PSR-7 request.

I've created a pseudo stream around mongodb cursor using the function futurTick() to allow the loop to retake controle.
So it's a get request that stream a long response and gives back the hand to the loop every 1000 rows so the server can handle other incoming request. Of course there is no reading on the incoming request and the onData trick to cancel the timer does note work. So I'd like to pass the timer object to the request layer so I could cancel it for this type og request.

@quazardous Thanks for bringing this up. I'm not sure I follow exactly what you're trying to achieve here, can you elaborate and/or provide a gist?
 This can be used as a response body using the existing APIs as described here: https://github.com/reactphp/http#streaming-outgoing-response
https://github.com/reactphp/http/issues/324 discusses adding additional attributes to the ServerRequestInterface object to access underlying TLS/HTTPS properties.
https://github.com/reactphp/http/issues/396 discusses ways to match the underlying TCP/IP connection to a HTTP request object by matching the source and destination addresses.
Do you feel there's anything missing here?

I'll try to reformulate.
The way the read timeout is working depends on (real) streams that the loop can check on.
client(POST/STREAM) ----> Loop(STREAM is alive) ----> response(BODY/STREAM) : OK

But in case of non stream request (Like GET)
client(GET) --X--> Loop(NO STREAM Left !!!) -----> response(Long response wrapped in timer / pseudo stream) : KO !!!!

What I need is a way to profile the timeouts at runtime because I handle different type of request:

Request A: (Real) Stream to (Real) Stream -> boring
Request B: Very short classic GET -> boring
Request C: Very long response on a GET (wrapped with timers/pseudo stream) -> special care

So it would be nice to add advanced timeout capabilities (not just a general-same-timeout-for-all).

 This can be used as a response body using the existing APIs as described here: https://github.com/reactphp/http#streaming-outgoing-response

In this case the on('data') ... timer trick cannot work because there is no stream left to read on the input side (aka Request C).
I'll try to dispatch my thoughts to the linked issues

Do you feel there's anything missing here?

the "advanced at runtime dynamic super timeout" could be discussed further

Your last gist implements a ReadableStreamInterface. This can be used as a response body using the existing APIs as described here: https://github.com/reactphp/http#streaming-outgoing-response

In this case the on('data') ... timer trick cannot work because there is no stream left to read on the input side (aka Request C).

The incoming response is indeed buffered by default to make it compatible with PSR-7. As per https://github.com/reactphp/http#streaming-incoming-request, you can explicitly stream the incoming response as well. In this case, you would have full control over managing the incoming stream and any timeouts yourself.
Other that that, I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can always reopen this :+1:


Create Wiki  pages for the project - Documentation

Wiki pages

overview
Features
DICOM Support
Development
...

Started building the wiki pages, still in progress

closing this as the wiki pages will be a living documentation that get improved overtime


Metadata: don't refresh periodically by default.

Currently driver refreshes metadata in 4 situations:

When requested by user
When receiving cql event that signals the need to refresh
When re-establishing control connection
Periodically, by default once every 60s. This can't be disabled (you can practically disable it by setting very long period).

Metadata may be big and so fetching it may take a lot of resources. This will be partially mitigated by https://github.com/scylladb/scylla-rust-driver/issues/595 because only necessary metadata will be fetched by default, but

Users who want to download full metadata will still need to pay the full cost
Even this limited metadata may be considered too expensive if downloaded unnecesarily

If cql-events are working correctly there should be no need to periodically refresh metadata. I propose to assume that they are working properly, make it possible to fully disable periodical fetching and disable it by default.

Something I was unaware of: https://github.com/scylladb/scylla-rust-driver/issues/546
Currently driver doesn't refresh on schema change event, only on topology change event.
This needs to be fixed before we can disable periodical fetching.


Make DoubledWord and DoubledJoshi deprecated

Fixed with https://github.com/redpen-cc/redpen/pull/592


LPC Summary - Green checkmark is shown for a tiprack that is not used in the protocol

Current Behavior
From Nusrat:
"I am running a protocol that has a p1000 tiprack being loaded but it is not being used in the protocol. In the deck map it is shown. However, during the final screen of LPC before applying labware offsets, there is a green check on it which indicates to me that the pipette has gone through the flow with that tiprack."
Expected Behavior
We should not show any labware that is not used in the protocol on the LPC summary screen.

Pushing this out of scope for launch. Let's look at this again when we're deciding on scope for 5.0.1

See if this is able to repo after 5.0.2
Bundle in LPC follow-ups?


Add wrapper for running apptainer on Mac

Now that Apptainer has been moved from experimental, and accepted as LF project, it might be time for a wrapper ?
There is no client for apptainer, similar to nerdctl.
So use a small shell wrapper, to run "lima apptainer".
This is so that the user can follow the documentation, without having to write an extra lima  prefix before every command:
https://apptainer.org/docs/user/main/quick_start.html
We might still have to document the read-only home.
FATAL:   While performing build: while creating SIF: while creating container: open /home/anders/lima/alpine.sif: read-only file system

This is different from the situation for docker and podman.
Since they already have client binaries, available for Mac.

It would be nice if this could automatically add the -B $HOME:$HOME mount, to run.


[WIP] Copy secret referenced by Account to secret that UHC expects

This grabs the secret holding the IAM info referenced by the account, and creates a secret with the same data with the name and namespace UHC expects via the accountclaim CR.
WIP because the code is untested

Just so you it is clear the secret UHC is expecting looks as follows:
kind: "Secret"
apiVersion: "v1"
metadata:
  name: secretName
  namespace: secretNamespace
data:
  aws_access_key_id: foo
  aws_secret_access_key: bar

@nimrodshn Is that secret structure documented somewhere?

@fahlmant Not really, this is what hive expects.


Missing RR implementation & advanced usage

Added support for MX, TXT, PTR, SOA, ANY, NS, AAAA records
Added additional & authority message parsers & dumper (includes RobinvdVleuten's https://github.com/reactphp/react/pull/272)
Fixed TCP retry parser & dumper. In TCP first two octets describe the length of data
Added explain method to Message, Query, Record for detailed information in a dig like format.
Added examples
Message->questions now stores array of Query objects instead of array of array

@igorw
What are your thoughts about merging it at this point and tackling state mutation later on as you originally suggested.

@igorw , @cboden
ping?

@clue Maybe something for you since you're also working on some more DNS PR's?

This PR does in fact contain quite a bit of useful code that I'd love to see included! Though I agree with @igorw this is a bit hard to review given the amount of code changed. Ideally, this would be split up into several smaller PRs.
Either way, is there anything that can be done to move this forward?

I would love to see this merged. Is there anything you need from me at this point @clue


Fixed the links and updated the UI of Refunds Page

Related Issue  #149 #147

Closes: #[issue number]
Description of Changes
->Fixed the Links of Internship Buttons
->Improved the UI of the Refunds Page
->Also included the footer in it.

Checklist:

[x] My code adheres to the established style guidelines of this project.
[x] I have conducted a self-review of my code.
[x] I have included comments in areas that may be difficult to understand.
[x] I have made corresponding updates to the project documentation.
[x] My changes have not introduced any new warnings.

Screenshots

Original
Updated

Note :
->The codebase for the Tnc Page and Privacy Policy Page is pretty messed up (without documentation) and would take a lot of time to fix. So I haven't done that

Could you pull latest changes and merge in this pr?

hey @divya-2910 ? R u up for it? if any issue you can ping

@anmode On it. I need some time

@anmode All changes done, check once and approve the PR

@anmode Returns policy page is already fixed in this PR #181

yeah! Actually in this pR. I was supposing to target all. privacy policy , refund policy and terms & Conditions.

@divya-2910 please fix the conficts...then i can review better

@anmode I think there are no conflicts now

why so many lines of changes ? @divya-2910

@AmanJain18 I think the TNC page was generated using some online generator.
So the person who wrote it forgot to copy the CSS part. Had to rewrite the design part from scratch using new classes and ids

@anmode
I made the font style same across all the three pages.  Also heading font style is the same as the index page heading in all 3 pages.
And I have fixed the anchor links in both pages


Update naming convention for v4: CSD-####-opt --> QMOF-####-opt

We will refer to all subsets of the QMOF database as QMOF-####-opt/SP instead of CSD-####-opt/SP starting in v4 and onward. All instances of this throughout the various READMEs and files will need to be updated.

Taken care of: https://github.com/arosen93/QMOF/commit/4ac378738ca4827d89ec16fb26446eeb6c81073e