[ES] add content/es/auto_scaling.md

PR for issue #565
Done.

@gamba47 Thank you for your contribution!
One should be able to see the preview of your work at https://deploy-preview-705--cncfglossary.netlify.app/es/auto_scaling/ ,
but I checked that I cannot.
It seems that this is because you started your work from main branch, not dev-es branch.
Referring this comment:

You can close this PR and fetch the upstream dev-es branch on your forked repo again, so you're aligned. Then start from scratch: open a new branch from the dev-es branch, add the new term, and raise another PR which aims the dev-es branch as a merge target.

Hi @jihoon-seo !   How can i change to the other branch?
I made 3 translations i will do more with this concept. Thanks for your time

@gamba47
This is how to create another PR:

On your development environment, pull all branches from upstream repo:
git pull
Switch to dev-es branch:
git checkout dev-es
Create a new feature branch (that will contain your new commit) from dev-es branch:
git checkout -b <new-branch-name>
Add a new file (content/es/auto_scaling.md), and commit:
git add content/es/auto_scaling.md (Add the version that @raelga's comments are reflected. 😊)
git commit -m "<Some commit message>"
Push your new feature branch to your forked GitHub repo (e.g. origin):
git push origin <new-branch-name>
On GitHub web UI, open a new PR, while setting the merge target branch as dev-es. 😊
After that, this PR gets okay to close. 😊

@gamba47 This is how to create another PR:
1. On your development environment, pull all branches from upstream repo:
   `git pull`

2. Switch to `dev-es` branch:
   `git checkout dev-es`

3. Create a new feature branch (that will contain your new commit) from `dev-es` branch:
   `git checkout -b <new-branch-name>`

What about to cherry-pick that change?
git pull
git checkout -b dev-es origin/dev-es
git branch -m auto_scalling draft
git branch auto_scalling
git checkout auto_scalling
git cherry-pick 7edd30c837d0f475e44395ff36e6b4c6b9ae0742
git push origin auto_scalling -f

4. Add a new file (`content/es/auto_scaling.md`), and commit:
   `git add content/es/auto_scaling.md` (Make sure that you add the version that @raelga's comments are reflected. 😊)
   `git commit -m "<Some commit message>"`

5. Push your new feature branch to your forked GitHub repo (e.g. origin):
   `git push origin <new-branch-name>`

6. On GitHub web UI, open a new PR, while setting the merge target branch as `dev-es`. 😊

7. After that, this PR gets okay to close. 😊

Let me know if you need a change in mi pull requests or if i do this on new pr only

Instead of open a new PR, updating this PR looks also good to me. 😊
Meanwhile, following @electrocucaracha's instruction, I got an error:
❯ git checkout -b dev-es origin/dev-es
fatal: 'origin/dev-es' is not a commit and a branch 'dev-es' cannot be created from it

For me, this worked (@electrocucaracha Please check whether below commands look good to you!):
git pull
git checkout dev-es
git branch -m auto_scalling draft
git branch auto_scalling
git checkout auto_scalling
git cherry-pick 7edd30c837d0f475e44395ff36e6b4c6b9ae0742
git push origin auto_scalling -f

Updating this means change where i merge my changes? sorry i'm a newbie in this git field.

@gamba47

Updating this means change where i merge my changes?

Yes, 'Updating this' means that you change the merge target branch of this PR to dev-es.
To do so, follow this:

On your development environment,

git pull
git checkout dev-es
git branch -m auto_scalling draft
git branch auto_scalling
git checkout auto_scalling
git cherry-pick 7edd30c837d0f475e44395ff36e6b4c6b9ae0742
git push origin auto_scalling -f

On this page, change the merge target branch of this PR to dev-es:

In this case, @raelga's changing the merge target branch to main only seems okay without problem.
(cf. #696)
Although the preview is not available on https://deploy-preview-705--cncfglossary.netlify.app/es/auto_scaling/
(which is because the Netlify rebuild is triggered by someone's pushing new commits to the branch),
I guess that this page will be displayed well after all.
@gamba47 @electrocucaracha @raelga Thank you for your contributions!

Closing and recreating another pr based on the last meeting


Spike: Fee summary data

The Chore
At launch / soon after we want to make sure that someone who has decided to trade, provide liquidity on Vega has an easy way to understand what they are making from fees and rewards. Right now, we have the ability to show reward scheme earnings in the token front end, but do not currently show fees:

Maker fees
LP fees
Infrastructure fees

Tasks

[ ] Identify the data required
[ ] Identify if this data is available via API
[ ] Outline ticket for creating the service or requirement on data node (if there is one needed)

Additional details / background info
These fees might be on the ledger entries API https://docs.vega.xyz/testnet/api/rest/data-v2/trading-data-service-list-ledger-entries- e.g.
TRANSFER_TYPE_MAKER_FEE_RECEIVE
TRANSFER_TYPE_LIQUIDITY_FEE_DISTRIBUTE
TRANSFER_TYPE_INFRASTRUCTURE_FEE_DISTRIBUTE

Need to make sure we include the historical granularity on this stuff.
Will people want to see by day, week, month, year?
How accurate within a day?  Is a day fine or does it need to be hourly?
When does a "day" start and end? Is epoch best? Maybe UTC? 00:00? Vegatime?
Really important we get the data engineering right on this first time!


fix(ci): use --start-load, not --load

--load means --start-load + --run-load, and --run-load actually slows down tests if the load of the system is too high, which is exactly what we want to avoid!

Nvmd, I've read docs for niceload by accident and got spooked.


Currently we are not able to use Safe Navigation

Description:
Currently, we are not able to use Safe Navigation. If you use safe navigation, an error of Unexpected token occurs.
example error:
ERROR in ./src/document/json/rga_tree_list.ts 33:30
Module parse failed: Unexpected token (33:30)
File was processed with these loaders:
 * ./node_modules/ts-loader/index.js
You may need an additional loader to handle the result of these loaders.
|     static createAfter(prev, value) {
|         const newNode = new RGATreeListNode(value);
>         const prevNext = prev?.next;
|         prev.next = newNode;
|         newNode.prev = prev;
 @ ./src/document/json/array.ts 17:0-46 32:40-51
 @ ./src/yorkie.ts

Why:
It seems to be a webpack issue. It seems to update the webpack version to 5 will fix it.
(For test environments, karam-webpack needs to be updated separately)
Reference: https://github.com/webpack/webpack/issues/10227

Safe navigation saves you the trouble of checking undefined.
// Before
    while (
      node!.getNext() &&
      (node!.getNext()!.getCreatedAt().after(executedAt) ||
        (node!.getNext()!.getValue().getMovedAt() &&
          node!.getNext()!.getValue().getMovedAt()!.after(executedAt)))
    ) {
      node = node!.getNext();
    }

// After
    while (
      node!.getNext()?.getCreatedAt().after(executedAt) ||
      node!.getNext()?.getValue().getMovedAt()!.after(executedAt)
    ) {
      node = node!.getNext();
    }


Pinout diagram for the LilyGo T-Display TOUCH version please

So sorry folks, I have seen this diagram but for the life of me I can't seem to find it any more.  I am sure I have seen a diagram on LilyGo GitHub [page somewhere that shows mainly the T-Display, but there is an added image on that page showing the additional Touch screen pinouts.  Could someone please put me out of my misery and point me to that page.
Ta.

do you mean this? https://github.com/Xinyuan-LilyGO/T-Display-S3/blob/main/schematic/T_Display_S3.pdf
Or something else?

do you mean this? https://github.com/Xinyuan-LilyGO/T-Display-S3/blob/main/schematic/T_Display_S3.pdf
Or something else?

Alas, something else @szucsz sorry.  I have seen a diagram of the pinout somewhere on their pages, but I just can't seem to locate it now!  Thanks for this though.

It was something like this one

But with an added diagram / image of the Touch version showing the pinouts.

https://github.com/Xinyuan-LilyGO/T-Display-S3
The ReadMe shows a pinout including the touch inputs.
Is this what you are talking about? GPIO 1-3 and 10-13.

No Terry.  I am certain I saw an image, which was the one in the readme file, but in the lower right hand side of the image there was an additional picture of the physical TOUCH version, with the pinout shown on that.

Is that what you mean?
https://github.com/Xinyuan-LilyGO/T-Display-S3/blob/main/image/T-DISPLAY-S3-TOUCH.png

Is that what you mean?
https://github.com/Xinyuan-LilyGO/T-Display-S3/blob/main/image/T-DISPLAY-S3-TOUCH.png

Yes, Lewis, that's the one.  Thank you very much.
And, I am certain that this was not on the GitHub page for LilyGo T-Display yesterday!  However, it is the image I was referring to, thanks.


Error:  attribute transform: Expected number, "translate(NaN, 0)".

When I generate my line series charts with the ajax ".load" function (getting json response), I get the following error in my js console.
Error:  attribute transform: Expected number, "translate(NaN, 0)".
How can I avoid it ? What's triggering this error ?

@jsncrdnl I noticed this happening when I hide the chart container using CSS, and resize the browser window. I suspect it has something to do with how c3 is calculating the size to apply to the chart.

In my example, I had an empty value in my tick values array (i.e tickValues=[empty, 300, 320] ), so it couldn't draw and gave the error. I fixed the values and it worked.


Issue 298 Language switch

This PR closes #304
This PR closes #298
This PR closes #301
Hopefully final rebased and squashed version of the language switch feature, including successful tests.

See https://github.com/DSpace/dspace-angular/pull/308 for the history and dialogue that lead to this final version.

Spoke too soon, still have a number of lint style issues to address, apologies.

Checks are passing now - I have currently no more changes pending here.


LDAP support is missing in helm chart

Hello,
The current release (1.0.4) of the helm chart does not support ldap authentication.
Please add this feature.

It feels that recent versions of this chart support passing in the values via environment variables.
https://github.com/bootc/netbox-chart/blob/master/values.yaml#L446-L452

This is implemented in #31 and will be in the 3.0.0 release of the chart which I plan to release this week.


Implementation updated jwt vc payload processing

This change allows us to consume a minimum viable VC JWT as well as a full featured VC JWT:
Minimum Viable VC JWT Example:
{
  "vc": {
    "@context": [
      "https://www.w3.org/2018/credentials/v1"
    ],
    "type": [
      "VerifiableCredential"
    ]
  },
  "iss": "did:jwk:eyJhbGciOiJFZERTQSIsImNydiI6IkVkMjU1MTkiLCJrdHkiOiJPS1AiLCJ4IjoiTDNjcngtRUlBMkUtbERyN2EtZ250UDdEbVVSMjJyWEg4eERRcWlUWHozMCJ9",
  "sub": "did:jwk:eyJhbGciOiJFZERTQSIsImNydiI6IkVkMjU1MTkiLCJrdHkiOiJPS1AiLCJ4IjoiTDNjcngtRUlBMkUtbERyN2EtZ250UDdEbVVSMjJyWEg4eERRcWlUWHozMCJ9",
  "exp":<PHONE_NUMBER>,
  "nbf":<PHONE_NUMBER>,
  "jti": "urn:vc:uuid:1ae664b9-9670-4be0-8d64-481df53dc05a"
}

we will accept this and dynaimcally build out the VC to fit the VC Data model standards using the claims. (for example vc.issuer will be iss)
We also accept and process a full featured VC JWT like this:
{
  "vc": {
    "@context": [
      "https://www.w3.org/2018/credentials/v1"
    ],
    "id": "urn:vc:uuid:dd05667d-8d82-429e-b32b-c557d20a4791",
    "type": [
      "VerifiableCredential"
    ],
    "issuer": "did:jwk:eyJhbGciOiJFZERTQSIsImNydiI6IkVkMjU1MTkiLCJrdHkiOiJPS1AiLCJ4IjoiVGR5SXdHRThlR2dIM3RvZFA0ak1mV0xSaE1kQkg4cW56aW4xZU5RUlpuMCJ9",
    "issuanceDate":<PHONE_NUMBER>,
    "expirationDate":<PHONE_NUMBER>,
    "credentialSubject": {
      "id": "did:jwk:eyJhbGciOiJFZERTQSIsImNydiI6IkVkMjU1MTkiLCJrdHkiOiJPS1AiLCJ4IjoiVGR5SXdHRThlR2dIM3RvZFA0ak1mV0xSaE1kQkg4cW56aW4xZU5RUlpuMCJ9"
    }
  },
  "iss": "did:jwk:eyJhbGciOiJFZERTQSIsImNydiI6IkVkMjU1MTkiLCJrdHkiOiJPS1AiLCJ4IjoiVGR5SXdHRThlR2dIM3RvZFA0ak1mV0xSaE1kQkg4cW56aW4xZU5RUlpuMCJ9",
  "sub": "did:jwk:eyJhbGciOiJFZERTQSIsImNydiI6IkVkMjU1MTkiLCJrdHkiOiJPS1AiLCJ4IjoiVGR5SXdHRThlR2dIM3RvZFA0ak1mV0xSaE1kQkg4cW56aW4xZU5RUlpuMCJ9",
  "exp":<PHONE_NUMBER>,
  "nbf":<PHONE_NUMBER>,
  "jti": "urn:vc:uuid:dd05667d-8d82-429e-b32b-c557d20a4791"
}

Suggestion for the CredentialError, that I probably should have brought up in an earlier PR:
Both CredentialError and CredentialDataModelValidationError have a lot of variants, and some of them aren't even used, which tells me that we should pare them both down a lot.
A lot of variants in CredentialError amount to "this field is missing" or "these fields mismatch". What do you think of consolidating each of those concepts into one error variant each? If you prefer, we can have MissingVcField and MissingJwtField be separate, but I think it would also be okay to just have one MissingField.
#[derive(thiserror::Error, Debug)]
pub enum CredentialError {
    // JWT specific errors
    ...

    // Credential-specific validation errors
    #[error("Credential expired: {0}")]
    CredentialExpired(String),
    #[error("Issuance date is in the future")]
    IssuanceDateInFuture,
    #[error("Misconfigured expiration date: {0}")]
    MisconfiguredExpirationDate(String),
    #[error("Mismatch between VC field {0} and JWT field {1}")]
    Mismatch(String, String),
    #[error("Missing VC field {0}")]
    MissingVcField(String),
    #[error("Missing JWT field {0}")]
    MissingJwtField(String).
}

I would also completely remove CredentialDataModelValidationError. Most of its variants have some equivalent in CredentialError and we only propagate CredentialDataModelValidationError wrapped inside of CredentialError::ValidationError.


Dot net 4.5 not supported?

I am getting following error when installing from Nuget
PM> Install-Package SqlKata -Pre
Installing 'SqlKata 1.0.0-beta-32'.
Successfully installed 'SqlKata 1.0.0-beta-32'.
Adding 'SqlKata 1.0.0-beta-32' to QueryBuilder.
Uninstalling 'SqlKata 1.0.0-beta-32'.
Successfully uninstalled 'SqlKata 1.0.0-beta-32'.
Install failed. Rolling back...
Install-Package : Could not install package 'SqlKata 1.0.0-beta-32'. You are trying to install this package into a project that targe
ts '.NETFramework,Version=v4.5', but the package does not contain any assembly references or content files that are compatible with t
hat framework. For more information, contact the package author.
At line:1 char:16

Install-Package <<<<  SqlKata -Pre

CategoryInfo          : NotSpecified: (:) [Install-Package], InvalidOperationException
FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PowerShell.Commands.InstallPackageCommand

@vickyrathee yes currently it is supported on netcoreapp1.0, a new release will be published soon targeting net45

My current focus is to support the netstandard 2.0, I appreciate if anyone could help on this.

Your library already works with netstandard2.0 projects.
But you could add net45 to the targetframeworks. I had to make these changes in the project file:
    <TargetFrameworks>netstandard1.3;net45</TargetFrameworks>

and had to add
  <ItemGroup Condition=" '$(TargetFramework)' == 'net45' ">
    <Reference Include="System" />
  </ItemGroup>

then it builds OK for me.

@ahmad-moussawi
Submitted PR for net45 (net452 for tests due to xunit)

@ahmad-moussawi
Since you have merged PR of @ceastwood , can you make a new release for net45

@SamarRizvi currently I don't have the needed env to test it for net45 since I am using a mac device, what actually needed is to close this issue:

Make sure that the tests passes on this environment
Configure AppVeyor to build the nuget packages after pushing a valid commit on the master branch
Make sure that this new release will not affect netcore projects

I think that @ceastwood is the best one for now to deal with this,
@ceastwood can you help in this ?

@ahmad-moussawi
I'm not at my dev machine right now but I do have travis-ci and appveyor ready for this repo, I can complete the tests-- I have additional uncommitted unit tests add as well. I can complete those and build the nuget packages locally but not push to your feed.
This change won't affect other projects if everything passes, but the PR I have for renaming will. Let me know what you would like and I can complete.

@ceastwood many thanks, mm to support net45 we have to automate the build for windows, I will check how we can do it with AppVeyor, and hope we may find someone who have better knowledge than me :)

@ahmad-moussawi
The appveyor image in use for this repo is hosted on Windows Server 2016;
I've created a new branch update-build-scripts on my fork of the repo and started to complete the tasks. I prefer to use a slightly different (but msbuild native) way of handling semver so that change is there too.
The changed I have pushed up to now in that branch builds and packages everything fine. I will create a matching build/pack shells scripts for you since you mentioned your on mac before I submit the PR.

I have net45 solution, not facing what @vickyrathee  had faced, but getting this:

Additional information: Could not load file or assembly 'System.Reflection.TypeExtensions, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

compiler.Compile(fastCarsQuery).Sql

@eyedia
Are you using the nuget package that is available? If so, that package states that it only supports netstandard1.3 as shown here (under dependencies)
Work is being done to update the build tooling for this project and when those changes are merged there should be a net45 nuget package available then.

Thanks @ceastwood , I missed that. Yes, it worked.
Wow! it installed 285MB packages, we definitely need net45 nuget package.
I also wanted to know how can I contribute, I have a sample database with various types of joins(relationship). I can provide queries/scenarios,/troubleshoot. let me copy @ahmad-moussawi  here.

@ceastwood yes sure, I tried to mimic Dapper build script build.ps1 but I've failed :s, I can add you as a member in AppVeyor if needed, I can help also in setting the shell script form my side

@eyedia your suggestions and contributions are always welcomed, currently as @ceastwood  mentioned we are focusing on targeting the 45 framework,
You can check https://github.com/sqlkata/querybuilder/projects/1 for our backlog

@ahmad-moussawi  I have no problem with the shell script, I just have been caught up with work. But I think appveyor will auto build when we commit to master; so the only way I can push things is with write access to this repo or with you merging the PR.

@ahmad-moussawi  I should have the build tooling update & PR done very soon, and I also have the rename PR I will resubmit after the first merge

Ok waiting the awesomeness 👍

@ahmad-moussawi Sorry, I took a little but of a break since I had to travel, but I should finish up by tomorrow. Thanks

@vickyrathee
@kwaclaw
@eyedia
@SamarRizvi
v1.0.0-beta-357 is published and supports a target framework of net45

Nice

@ceastwood Thanks a lot.


Recognize festive season

This module with the different festive seasons would be a good candidate to activate itself a few days before the season.
Having the calendar module usually installed, it may be possible to recognize e.g. christmas via a notification by calendar or just via a hardcoded date and activate the module a configurable range of days before that.
It would even be possible to show the NFL team colors automatically on the day the team plays.

I agree with @lavolp3 .
My suggestion would be to map the seasonal css with a date range that user will set.
For example, I could decide that the christmas.css will be displayed from Dec. 1st to January 7th.

I agree with @lavolp3 .
My suggestion would be to map the seasonal css with a date range that user will set.
For example, I could decide that the christmas.css will be displayed from Dec. 1st to January 7th.


Research Support for Future T-Mobile Gateways

Per https://tmo.report/2021/12/this-is-the-new-t-mobile-5g-home-internet-gateway/ there is likely a new 5G Gateway coming for T-Mobile Home Internet. Ideally this script should be compatible with more than just the Nokia 5G Gateway.

The Arcadyan KVD21 Gateway has begun shipping to customers. Not much is known yet besides the following:

The Web UI only displays the number of bars for the connection information.
The Mobile app displays the CGI as the concatenation of the T-Mobile PLMN and the Hex Representation of the rest of the number
The Mobile app does not indicate which band is connected

KVD21 does provide signal info at the following unauthenticated WEB UI endpoints:

http://<IP_ADDRESS>/TMI/v1/gateway?get=all
http://<IP_ADDRESS>/TMI/v1/gateway?get=signal

This includes 4G and 5G bands, RSSI, RSRP, RSRQ, SINR
The first endpoint also includes the gateway uptime and software version.
The mobile app endpoints appear to require likely oAuth-based token (Bearer) authentication and exposes more information, including:

PLMN, CGI, 4G and 5G bandwidth/channel size, EARFCN/ARFCN (frequency)
GPS latitude and longitude
List of clients separated into 2.4GHz, 5GHz, and ethernet, with hostname, signal strength (for WLAN clients), MAC, IPv4, and v6 addresses

IT also appears to include a list of neighboring cells, albeit with inaccurate metric values.
Both web UI and app endpoints appear to have incorrect eNB ID and gNB ID values.

Closed by #47


Missing sections in diff viewer

Sometimes the diff has missing sections

diffmissing.txt

Not seeing this bug anymore


Reorganize the BigQuery datasets to be more efficient

Similar to the almanac dataset, we want the monthly results to be queryable in partitioned and clustered tables.
We'll need a deprecation plan to avoid suddenly breaking existing queries:

[x] create the new tables

[x] #75
[x] #109

[x] backfill with existing data
[x] alter the pipeline to write to the new tables in addition to the existing tables https://github.com/HTTPArchive/data-pipeline/issues/38
[x] document how to query the new tables
[x] announce the deprecation plan
[ ] migrate existing queries to the new tables (i.e. for generate_reports)
[x] wait N months to allow the community to upgrade their queries
[x] stop writing to the old tables in the pipeline
[ ] delete the old tables

Brainstorming new tables/schemas.

httparchive

har

pages
requests

httparchive.har.pages
All-in-one table containing all page-based results from all crawls for all clients.
Partition by:

date

Cluster by:

client
is_root_page
rank

Combines data from multiple existing tables:

pages
summary_pages
blink_features
technologies
lighthouse

Additional enhancements:

New custom_metrics field
features and technologies fields are well-defined repeated structs

Combining categories into a single repeated STRING field, rather than creating new technology+category pairs, which tripped up many users (including myself)

summary field is JSON-encoded so we can support all historical summary_pages stats if we'd like and add/remove anything over time without breaking the schema

[
    {
        "name": "date",
        "type": "DATE",
        "mode": "REQUIRED",
        "description": "YYYY-MM-DD format of the HTTP Archive monthly crawl"
    },
    {
        "name": "client",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Test environment: desktop or mobile"
    },
    {
        "name": "page",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "The URL of the page being tested"
    },
    {
        "name": "is_root_page",
        "type": "BOOLEAN",
        "mode": "NULLABLE",
        "description": "Whether the page is the root of the origin."
    },
    {
        "name": "rank",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "Site popularity rank, from CrUX"
    },
    {
        "name": "wptid",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "ID of the WebPageTest results"
    },
    {
        "name": "payload",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "JSON-encoded WebPageTest results for the page"
    },
    {
        "name": "summary",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "JSON-encoded summarization of the page-level data"
    },
    {
        "name": "custom_metrics",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "JSON-encoded test results of the custom metrics"
    },
    {
        "name": "lighthouse",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "JSON-encoded Lighthouse report"
    },
    {
        "name": "features",
        "type": "RECORD",
        "mode": "REPEATED",
        "description": "Blink features detected at runtime (see https://chromestatus.com/features)",
        "fields": [
            {
                "name": "feature",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Blink feature name"
            },
            {
                "name": "id",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Blink feature ID"
            },
            {
                "name": "type",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Blink feature type (css, default)"
            }
        ]
    },
    {
        "name": "technologies",
        "type": "RECORD",
        "mode": "REPEATED",
        "description": "Technologies detected at runtime (see https://www.wappalyzer.com/)",
        "fields": [
            {
                "name": "technology",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Name of the detected technology"
            },
            {
                "name": "categories",
                "type": "STRING",
                "mode": "REPEATED",
                "description": "List of categories to which this technology belongs"
            },
            {
                "name": "info",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Additional metadata about the detected technology, ie version number"
            }
        ]
    }
]

httparchive.har.requests
All-in-one table containing all request-based results for all crawls and clients.
Partition by:

date

Cluster by:

client
is_main_document
type

Combines data from multiple existing tables:

requests
summary_requests

Additional enhancements:

Top-level request_headers and response_headers fields that are well-defined repeated structs of key/value pairs, borrowing from the almanac.summary_response_bodies table
summary field is JSON-encoded so we can support all historical summary_requests stats if we'd like and add/remove anything over time without breaking the schema

[
    {
        "name": "date",
        "type": "DATE",
        "mode": "REQUIRED",
        "description": "YYYY-MM-DD format of the HTTP Archive monthly crawl"
    },
    {
        "name": "client",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Test environment: desktop or mobile"
    },
    {
        "name": "page",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "The URL of the page being tested"
    },
    {
        "name": "url",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "The URL of the request"
    },
    {
        "name": "is_base_page",
        "type": "BOOLEAN",
        "mode": "REQUIRED",
        "description": "Whether this request corresponds with the main HTML document of the page, which is the first HTML request after redirects"
    },
    {
        "name": "type",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "Simplified description of the type of resource (script, html, css, text, other, etc)"
    },
    {
        "name": "index",
        "type": "INTEGER",
        "mode": "NULLABLE",
        "description": "The sequential 0-based index of the request"
    },
    {
        "name": "payload",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "JSON-encoded WebPageTest result data for this request"
    },
    {
        "name": "summary",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "JSON-encoded summarization of request data"
    },
    {
        "name": "request_headers",
        "type": "RECORD",
        "mode": "REPEATED",
        "description": "Request headers",
        "fields": [
            {
                "name": "name",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Request header name"
            },
            {
                "name": "value",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Request header value"
            }
        ]
    },
    {
        "name": "response_headers",
        "type": "RECORD",
        "mode": "REPEATED",
        "description": "Response headers",
        "fields": [
            {
                "name": "name",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Response header name"
            },
            {
                "name": "value",
                "type": "STRING",
                "mode": "NULLABLE",
                "description": "Response header value"
            }
        ]
    },
    {
        "name": "response_body",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "Text-based response body"
    }
]

Generating May 2022 data for httparchive.all.pages:
CREATE TEMP FUNCTION GET_CUSTOM_METRICS(payload STRING) RETURNS STRING LANGUAGE js AS '''
const $ = JSON.parse(payload);
return JSON.stringify(Object.fromEntries($._custom.map(name => {
  let value = $[`_${name}`];
  if (typeof value == 'string') {
    try {
      value = JSON.parse(value);
    } catch (e) {
      // The value is not a JSON string.
    }
  }
  return [name, value];
})));
''';

CREATE TEMP FUNCTION GET_FEATURES(payload STRING)
RETURNS ARRAY<STRUCT<feature STRING, id STRING, type STRING>> LANGUAGE js AS
'''
  function getFeatureNames(featureMap, featureType) {
    try {
      return Object.entries(featureMap).map(([key, value]) => {
        // After Feb 2020 keys are feature IDs.
        if (value.name) {
          return {'feature': value.name, 'type': featureType, 'id': key};
        }
        // Prior to Feb 2020 keys fell back to IDs if the name was unknown.
        if (idPattern.test(key)) {
          return {'feature': '', 'type': featureType, 'id': key.match(idPattern)[1]};
        }
        // Prior to Feb 2020 keys were names by default.
        return {'feature': key, 'type': featureType, 'id': ''};
      });
    } catch (e) {
      return [];
    }
  }

  var $ = JSON.parse(payload);
  if (!$._blinkFeatureFirstUsed) return [];

  var idPattern = new RegExp('^Feature_(\\d+)$');
  return getFeatureNames($._blinkFeatureFirstUsed.Features, 'default')
    .concat(getFeatureNames($._blinkFeatureFirstUsed.CSSFeatures, 'css'))
    .concat(getFeatureNames($._blinkFeatureFirstUsed.AnimatedCSSFeatures, 'animated-css'));
''';

WITH pages AS (
  SELECT
    _TABLE_SUFFIX AS client,
    url AS page,
    JSON_VALUE(payload, '$._metadata.rank') AS rank,
    JSON_VALUE(payload, '$._metadata.crawl_depth') = '0' AS is_root_page,
    JSON_VALUE(payload, '$._testID') AS wptid,
    GET_CUSTOM_METRICS(payload) AS custom_metrics,
    JSON_QUERY(payload, '$._metadata') AS metadata,
    payload
  FROM
    `httparchive.pages.2022_05_01_*`
# TODO: Backfill when the summary pages is ready.
/* ), summary_pages AS (
  SELECT
    _TABLE_SUFFIX AS client,
    url AS page,
    TO_JSON_STRING(summary_pages) AS summary
  FROM
    `httparchive.summary_pages.2022_05_01_*` AS summary_pages */
), loose_technologies AS (
    SELECT
      _TABLE_SUFFIX AS client,
      url AS page,
      STRUCT(
        app AS technology,
        ARRAY_AGG(DISTINCT category ORDER BY category) AS categories,
        ARRAY_AGG(info) AS info
      ) AS technology
    FROM
      `httparchive.technologies.2022_05_01_*`
    GROUP BY
      client,
      page,
      app
), techs AS (
  SELECT
    client,
    page,
    ARRAY_AGG(technology) AS technologies
  FROM
    loose_technologies
  GROUP BY
    client,
    page
), lh AS (
  SELECT
    _TABLE_SUFFIX AS client,
    url AS page,
    report AS lighthouse
  FROM
    `httparchive.lighthouse.2022_05_01_*`
)

SELECT
  DATE('2022-05-01') AS date,
  client,
  page,
  is_root_page,
  rank,
  wptid,
  payload,
  # TODO: Update when the summary pipeline completes successfully.
  '' AS summary,
  custom_metrics,
  lighthouse,
  GET_FEATURES(payload) AS features,
  technologies,
  metadata
FROM
  pages
LEFT JOIN
  techs
USING
  (client, page)
LEFT JOIN
  lh
USING
  (client, page)

Pages data for April 2022. Changed rank to an INT64 field and added the required root_page field.
CREATE TEMP FUNCTION GET_CUSTOM_METRICS(payload STRING) RETURNS STRING LANGUAGE js AS '''
const $ = JSON.parse(payload);
return JSON.stringify(Object.fromEntries($._custom.map(name => {
  let value = $[`_${name}`];
  if (typeof value == 'string') {
    try {
      value = JSON.parse(value);
    } catch (e) {
      // The value is not a JSON string.
    }
  }
  return [name, value];
})));
''';

CREATE TEMP FUNCTION GET_FEATURES(payload STRING)
RETURNS ARRAY<STRUCT<feature STRING, id STRING, type STRING>> LANGUAGE js AS
'''
  function getFeatureNames(featureMap, featureType) {
    try {
      return Object.entries(featureMap).map(([key, value]) => {
        // After Feb 2020 keys are feature IDs.
        if (value.name) {
          return {'feature': value.name, 'type': featureType, 'id': key};
        }
        // Prior to Feb 2020 keys fell back to IDs if the name was unknown.
        if (idPattern.test(key)) {
          return {'feature': '', 'type': featureType, 'id': key.match(idPattern)[1]};
        }
        // Prior to Feb 2020 keys were names by default.
        return {'feature': key, 'type': featureType, 'id': ''};
      });
    } catch (e) {
      return [];
    }
  }

  var $ = JSON.parse(payload);
  if (!$._blinkFeatureFirstUsed) return [];

  var idPattern = new RegExp('^Feature_(\\d+)$');
  return getFeatureNames($._blinkFeatureFirstUsed.Features, 'default')
    .concat(getFeatureNames($._blinkFeatureFirstUsed.CSSFeatures, 'css'))
    .concat(getFeatureNames($._blinkFeatureFirstUsed.AnimatedCSSFeatures, 'animated-css'));
''';

WITH pages AS (
  SELECT
    _TABLE_SUFFIX AS client,
    url AS page,
    SAFE_CAST(JSON_VALUE(payload, '$._metadata.rank') AS INT64) AS rank,
    COALESCE(JSON_VALUE(payload, '$._metadata.crawl_depth') = '0', TRUE) AS is_root_page,
    JSON_VALUE(payload, '$._testID') AS wptid,
    GET_CUSTOM_METRICS(payload) AS custom_metrics,
    JSON_QUERY(payload, '$._metadata') AS metadata,
    payload
  FROM
    `httparchive.pages.2022_04_01_*`
), summary_pages AS (
  SELECT
    _TABLE_SUFFIX AS client,
    url AS page,
    rank,
    TO_JSON_STRING(summary_pages) AS summary
  FROM
    `httparchive.summary_pages.2022_04_01_*` AS summary_pages
), loose_technologies AS (
    SELECT
      _TABLE_SUFFIX AS client,
      url AS page,
      STRUCT(
        app AS technology,
        ARRAY_AGG(DISTINCT category ORDER BY category) AS categories,
        ARRAY_AGG(info) AS info
      ) AS technology
    FROM
      `httparchive.technologies.2022_04_01_*`
    GROUP BY
      client,
      page,
      app
), techs AS (
  SELECT
    client,
    page,
    ARRAY_AGG(technology) AS technologies
  FROM
    loose_technologies
  GROUP BY
    client,
    page
), lh AS (
  SELECT
    _TABLE_SUFFIX AS client,
    url AS page,
    report AS lighthouse
  FROM
    `httparchive.lighthouse.2022_04_01_*`
)

SELECT
  DATE('2022-04-01') AS date,
  client,
  page,
  is_root_page,
  COALESCE(pages.rank, summary_pages.rank) AS rank,
  wptid,
  payload,
  summary,
  custom_metrics,
  lighthouse,
  GET_FEATURES(payload) AS features,
  technologies,
  metadata
FROM
  pages
LEFT JOIN
  summary_pages
USING
  (client, page)
LEFT JOIN
  techs
USING
  (client, page)
LEFT JOIN
  lh
USING
  (client, page)

Ran into OOM issues with generating the requests table directly in BQ.
Since we'll eventually need to generate tables in the new all dataset from Dataflow, I decided to prototype how that pipeline would work. See https://github.com/HTTPArchive/bigquery/pull/170. Summary data is omitted for now since we'll be merging pipelines soon anyway.
I've successfully tested it on a single HAR and now attempting a full-scale test on the entire 2022_05_12 crawl. I expect it to take 4 hours (running for ~2 so far).

is_main_document is equivalent to the old firstHtml field, so having that parity might make migrating to the new schema easier. But I'm happy to explore other faster/cheaper schemas. Would you be willing to create a temp table with the proposed schema and benchmark a few test queries against each?

If I query for
SELECT
  JSON_VALUE(lighthouse, '$.finalUrl') AS final_url,
FROM `httparchive.all.pages`
WHERE
  date = "2022-10-01"
  AND client = 'mobile'
  AND is_root_page

The BQ UI warns me I'm going to be querying 13.65 TB, vs 4.63 TB for the (currently) equivalent
SELECT
  JSON_VALUE(report, '$.finalUrl') AS final_url,
FROM `httparchive.lighthouse.2022_10_01_mobile`

Is there some partitioning/clustering trick I should be doing to bring the all number more in line with the lighthouse number? Or more that could be done to the table structure? That would be an unfortunate cost increase for querying the raw data.

huh, so weird. Pasting into a fresh editor, same 13.65 TB

could being a member of the project somehow affect that? I don't see how, but not sure what other difference there could be.

The old BQ estimation behavior was to not take clustered fields into consideration. I'm also seeing the new behavior showing the 4.67 TB estimate, so maybe they've only partially rolled it out.

via the CLI:
bq query --nouse_legacy_sql --dry_run 'SELECT
  JSON_VALUE(lighthouse, "$.finalUrl") AS final_url,
FROM `httparchive.all.pages`
WHERE
  date = "2022-10-01"
  AND client = "mobile"
  AND is_root_page'

also gives me "this query will process upper bound of<PHONE_NUMBER>0052 bytes of data" (aka 13.65 TiB). Can either of you try it with a secondary cloud project to see if you get the same numbers?
If everyone eventually gets the lower number, obviously this isn't an actual issue.

I see 13.65TB when using a random account. Hmm....

If you run the query it should still only process 4.67 TiB

Well, here is the current plan for this reorganization: https://docs.google.com/document/d/1kNCpVgvC0W77HJzXiNvlchuGjCCY6uymIfv0nH10tF0/edit

Closing as most of the steps were implemented.
We have planned legacy data access deprecation (and cleanup) in March 2025.
Migration of the reports pipeline is pending: https://github.com/HTTPArchive/httparchive.org/issues/938


Update api-server-vnet-integration.md

Added NSG rules required to enable communication between components.

Could you please review this change @chasewilson @feiskyer

Could you please review this change @chasewilson @feiskyer

@asudbring
Can you review the proposed changes?
Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

@rahulrai-in @asudbring - the table was broken so I changed the markdown syntax to fix the issue, but please check that this is what you intended.:

Destination
Source
Protocol
Port
Use

APIServer Subnet CIDR
Cluster Subnet
TCP
443 and 4443
Required to enable communication between Nodes and the API server.

APIServer Subnet CIDR
Azure Load Balancer
TCP
9988
Required to enable communication between Azure Load Balancer and the API server. You can also enable all communicatios between the Azure Load Balancer and the API Server Subnet CIDR.

If it was NOT, we can fix in a new PR. thanks.


default queries to unfiltered

Because the query builder for the Node.js API generates query options, it can't rely on the default query options maintained by the REST API.
Thus, the query builder must generate the unfiltered option by default.

If the query doesn't have a .withOptions{search:['filtered']} clause, the query defaults to unfiltered.

Test added in regression under nodejs-documents-query.js


[UI/Figma] Improve the Card UI

Is your feature request related to a problem? Please describe.
The Card UI is currently looking bland and need to be revamped.
Describe the solution you'd like
The new design should first be shown in Figma and then implemented.
Describe alternatives you've considered
If you just want to design in Figma and do not want to be bothered with the code, you can do that but the difficulty will be changed to easy.

Pls assign this issue to me under JWOC

Is this design fine?

Yes, Can you change the airing sign a bit, or keep it as it is in the app ?

Any update ?

Will be done by tomorrow


Added pull_request_template

Added pull_request_template (Should this be merged to master or here?)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Cannot edit card

So, I want to edit my card(#410) from Idolized, Max Bonded to Idolized, Max Bonded, and Max Leveled. But everytime I click the edit card button, there's no popup to edit the card info.
Tired full reloading the page, still same.

What's your browser? Can you open the javascript console and tell me what you see? https://developer.chrome.com/devtools/docs/console#opening-the-console

I use Mozilla Firefox 38. When I open Developer Console yesterday, I see some javascript errors. Not sure for the exact error because I can't access the site this time(gives me Bad Request :confused: )

It seems that I can't edit my card info from the card page itself(example: I can't edit my card info from here) but I can edit my card info from /cards/ page(and display all of my cards from there).

Okay previously everything working fine (I believe it working fine in October) but I can't edit the card info again, both from the /cards/ page and from the /cards// page.


I can't for the life of me figure out why I can create a new user with a POST request, and although the user is actually created in the database, the Controller returns saying:
<!DOCTYPE html>

	<meta charset="utf-8">
	<title>Error</title>

Cannot POST /v1/user

I make a request (POST localhost:4000/v1/user) with the data { email, name, password} and although the user is created in the database i get the above response.
Example
UserController

@Controller("/user")
class UserController {

    constructor(private userService: UserService) {}

    @Post("/")
    @Status(201)
    public async create(@BodyParams() account: User): Promise<void> {
        await this.userService.createOne(account);
    }

    ...

}

UserService
@Service()
class UserService extends CRUDService<User> {

    @Inject(ORMService)
    protected ORM: ORMService;
    protected key = "email";
    protected model = User;

    public async createOne(account: User): Promise<User> {
        return await super.createOne({
            ...account,
            password: bcrypt.hashSync(account.password, 4)
            createdAt: new Date(),
            updatedAt: new Date()
        });
    }

    ...

}

CRUDService
@Service()
export class CRUDService<M> {

    @Inject(ORMService)
    protected ORM: ORMService;

    protected model: any;
    protected key: string;

    public async createOne(obj: M): Promise<any> {
        return this.ORM.getConnection().getRepository<M>(this.model).save(obj)
    }

    // 80 lines of createOne/Many, updateOne/Many, getOne/All & deleteOne/Many.

}

The problem is probably about the return value on your controller.
You don't return a value but you use a 201 status code (this is not an empty content). Try to add a return value:
@Controller("/user")
class UserController {

    constructor(private userService: UserService) {}

    @Post("/")
    @Status(201)
    public async create(@BodyParams() account: User): Promise<void> {
        return this.userService.createOne(account);
    }

    ...

}

Thank's for the quick response. Unfortunately using a 204 doesn't work either. I tried both 204 & 201 with and without returning a value. Is it worth mentioning I'm using sockets as well? It's probably irrelevant though.

This is the$beforeRoutesInit method in my ServerLoader class.
public $beforeRoutesInit(): void | Promise<any> {
    const bodyParser = require("body-parser");
    this.use(cookieParser())
    .use(compress({}))
    .use(cors())
    .use(methodOverride())
    .use(bodyParser.json())
    .use(bodyParser.urlencoded({
        extended: true
    }));
    this.use(
        secret: process.env.SESSION_SECRET,
        resave: false,
        saveUninitialized: true,
        cookie: {secure: true}
    }));
    return null;
}

Solved the issue. It turns out it was related to my sockets aha. I was using @Override(SendResponseMiddleware) instead of @SocketMiddleware() which seems to have overwritten the responses from my REST api response middleware.
Thank's for the help :)


CondaVerificationError: for "go-cgo" package

Issue: after call conda install go-cgo -c conda-forge I got next error:
CondaVerificationError: The package for go located at /disk/user/miniconda3/pkgs/go-1.13.10-hd4bb31f_2
appears to be corrupted. The path 'go/test/fixedbugs/issue27836.dir/foo.go'
specified in the package manifest cannot be found.

CondaVerificationError: The package for go located at /disk/user/miniconda3/pkgs/go-1.13.10-hd4bb31f_2
appears to be corrupted. The path 'go/test/fixedbugs/issue27836.dir/main.go'
specified in the package manifest cannot be found.

Details about  conda and system ( conda info ):

$ conda info
     active environment : base
          conda version : 4.8.3
    conda-build version : not installed
         python version : 3.7.4.final.0
       virtual packages : __glibc=2.30
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
               platform : linux-64
             user-agent : conda/4.8.3 requests/2.23.0 CPython/3.7.4 Linux/5.3.0-26-generic ubuntu/19.10 glibc/2.30
             netrc file : None
           offline mode : False

Proposal:
After fast search I found line, that removed problem directory for Windows https://github.com/conda-forge/go-feedstock/blob/d447bb4d14b77c7fd1de969da2464f300aa11816/recipe/build-base.bat#L34
Most likely we should add the removal of this directory into the build-base.sh

@anmyachev, thanks for reporting the bug, I will cut a new release without that offending directory.
It is unclear why that is going on, but I imagine it is an issue with libarchive.


Translate(x,y) Animation

Looking for a some help here. Of all the animation libraries I have looked at, not able to find one which provides an easy translate(x,y) animation. Are there are any plans to add it, will really make this library a complete one.
Something like this  http://jsfiddle.net/doktormolle/adcvK/
Thanks for the amazing work on this till now.

I like that. Right now I'm making the library more modular so creating new animations would be trivial.


Create a Graph section (graph-01.html

One need to make this component using HTML and Tailwind CSS. I would suggest to use Tailwind Playgrounds to make things faster and quicker.
Here is a reference to the component.

After building the component please raise a PR with a screenshot of the component and add the component in seamless-ui/Graph/src/.
If you need to use any icon please use it from Hero Icons
Good luck.
The graph section UI will help user to visualize the data more clearly.
@Rajdip019

Add it to others section. @G0V1NDK

Okay. I will do the same.


Feature Request: Theming

First a big thank you: You solved all my problems I had with Swagger: Swagger UI. Your project is really awesome and I enjoyed to read the source code of such a high quality with a very good code coverage. It just works for me.
I just have one small feature request: I would like to setup some theming properties in the redoc element: The logo and the accent colors would be fine. I can easily fix the colors with css but the logo is a "small" problem, because not all swagger tools support vendor extensions.

@SebastianStehle thanks for the request.
It will definitely be implemented soon

The 2.0.0-alpha.x version of ReDoc has theme configuration option which allows configuration of basic theme values.
Closing the issue.
If the new theming is not as good, open separate issues.
Thanks!


[2053] Clear ethnic background when not selecting group

Context
https://trello.com/c/xqMUFZy8/2053-bug-when-un-assigning-fields-in-trainees-we-assign-the-empty-fields-strings-instead-of-nil
The ethnic background question is hidden when you select (not provided)
as an ethnic group. So previously, if you selected an ethnic group and
background, but then changed the group to not provided, the result
on the confirmation page would be something like:
Not provided (Irish traveller) without giving you a way to clear it.
Changes proposed in this pull request
Set ethnic_background and additional_ethnic_background to nil when ethnic_group is not provided
Guidance to review

 [Technical Debt] When un-assigning fields in Trainees, we assign the empty fields strings instead of nil


s/Kill/Killed/ in storage.kill

This PR fixes a bug with killing a job.
Bug details
In MySQL/Postgres/Sqlite3 backends, Storage::kill(self, worker_id, job_id) sets the status to 'Kill', which is incompatible to the definition of JobStatus::Killed.
https://github.com/geofmureithi/apalis/blob/3cfc8df9a9bcb288ea24c3892e2446c45247b947/packages/apalis-core/src/request.rs#L12-L26
As a result, deserialization of job context will panic while converting Kill to JobState. https://github.com/geofmureithi/apalis/blob/3cfc8df9a9bcb288ea24c3892e2446c45247b947/packages/apalis-sql/src/from_row.rs#L53

This looks good 👏👏,but looking at that file and may others, seems I added a lot of unwraps, it might be worth to review this, maybe in another issue/discussion.


BrowserAnimationsModule should be imported only once by the using app's root app.module

This fixes the error, "BrowserModule has already been loaded" by an application that uses this package
(in my case, Angular 10.2.3)

It would be nice if it has merged soon because it is an annoying bug.


WIP: Sign systemd boot EFI images for secure booting.

with efitools (nix-shell -p efitools)
and running with openssl in the shell environment:
$ cd /home/grahamc/projects/grahamc/secure-boot
$ uuidgen --random > GUID.txt
$ openssl req -newkey rsa:4096 -nodes -keyout PK.key -new -x509 -sha256 -days 3650 -subj "/CN=my Platform Key/" -out PK.crt
$ openssl x509 -outform DER -in PK.crt -out PK.cer
$ cert-to-efi-sig-list -g "$(< GUID.txt)" PK.crt PK.esl
$ sign-efi-sig-list -g "$(< GUID.txt)" -k PK.key -c PK.crt PK PK.esl PK.auth
$ sign-efi-sig-list -g "$(< GUID.txt)" -c PK.crt -k PK.key PK /dev/null rm_PK.auth

$ openssl req -newkey rsa:4096 -nodes -keyout KEK.key -new -x509 -sha256 -days 3650 -subj "/CN=my Key Exchange Key/" -out KEK.crt
$ openssl x509 -outform DER -in KEK.crt -out KEK.cer
$ cert-to-efi-sig-list -g "$(< GUID.txt)" KEK.crt KEK.esl
$ sign-efi-sig-list -g "$(< GUID.txt)" -k PK.key -c PK.crt KEK KEK.esl KEK.auth

$ openssl req -newkey rsa:4096 -nodes -keyout db.key -new -x509 -sha256 -days 3650 -subj "/CN=my Signature Database key/" -out db.crt
$ openssl x509 -outform DER -in db.crt -out db.cer
$ cert-to-efi-sig-list -g "$(< GUID.txt)" db.crt db.esl
$ sign-efi-sig-list -g "$(< GUID.txt)" -k KEK.key -c KEK.crt db db.esl db.auth

and then loading the .auth files in my UEFI firmware, then setting in
my configuration.nix:
boot = {
  loader = {
    systemd-boot = {
      enable = true;
      signed = true;
      signing-key = "/home/grahamc/projects/grahamc/secure-boot/db.key";
      signing-certificate = "/home/grahamc/projects/grahamc/secure-boot/db.crt";
    };
  };
};

and running nixos-rebuild boot I was able to secure-boot my laptop.
Note: this code signs EVERY boot entry for NixOS, and the BOOT files
from systemd (even if they're already tampered with! since this code
doesn't replace it.) AND does not allow a graceful key phase out. If
the key is replaced in gen2, gen1's boot EFI is deleted and replaced
with a new version.
Also note this only works on x86-64 due to the assumptions in the
builder.py.
Finally, you can have much fewer generations: each generation on my
laptop takes 20MB in /boot.
Info taken from:

https://wiki.archlinux.org/index.php/Secure_Boot#Using_your_own_keys
https://systemd.io/BOOT_LOADER_SPECIFICATION#type-2-efi-unified-kernel-images
https://www.freedesktop.org/software/systemd/man/os-release.html

@GrahamcOfBorg eval

:exclamation: SO EXCITE :exclamation:

Any progress on this?

I think Graham wanted some people to test this code before he would merge it.

I'm now testing this on my machines, and so far it's working perfectly.
One idea I did have to maybe make the process more straightforward is to by default have a location for the certificates and keys in /var, which a oneshot service populates if it does not exist (somewhat like how letsencrypt certificates are automatically requisitioned). That said, users would still need to know where to find certs, and how to install them in their BIOS, so I'm not sure if it's worth it.

Working fine on my 9570.

@tkerber generating certificates sounds like a good idea. The chain of commands looks complex enough to make it a timer saver.

I've recently tested on HP Microserver Gen10 with positive result. sbsign whines several times that "gaps between PE/COFF sections?", but apparently it's not a problem.

If @grahamc does have no time to work on this PR, somebody else can take it over. I would review it.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/cli-to-plasma-installation-docs-or-help/4631/6

I'd love to see this PR upstream, i wanted to set up my laptop with secure boot for a long time

Any progress on this?

Taking a look.

@grahamc Any results from your look? Thanks!

Friendly reminder to look at this again. :smiley:

I took another look. I got stumped at wanting to test this automatically in a VM test and I wasn’t sure how to generate keys and register it with Qemu.
On Sat, Apr 11, 2020, at 2:15 PM, Bruno Bigras wrote:

Friendly reminder to look at this again. 😃

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/NixOS/nixpkgs/pull/53901#issuecomment-612481616, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCEZ2VQ7EWAZUSOV7LV6RTRMCXVTANCNFSM4GPVW2YA.

@grahamc I submitted a PR for testing systemd-boot, which required fixing some issues with installing the bootloader in the testing environment. It might prove useful for testing secure boot as well.

Just reposting as me (oops):
I took another look. I got stumped at wanting to test this automatically in a VM test and I wasn’t sure how to generate keys and register it with Qemu.

@grahamc I had some free time this weekend and decided to take a stab at making a test for this: https://github.com/danielfullmer/nixpkgs/commit/a2b95a3caa79f05dbab4ef06f7182bf7d4725099
To easily test:
nix-build "https://github.com/danielfullmer/nixpkgs/archive/a2b95a3caa79f05dbab4ef06f7182bf7d4725099.tar.gz" -A nixosTests.systemd-boot.secureBoot

It uses ovmf-vars-generator to load a certificate into nvram and enable secure boot. It still definitely needs some reorganization, but I'm out of time today and I finally got it working--so I thought I might share. :)

This is amazing! Thank you a lot, @danielfullmer. That should make it a lot easier to get a test going! :grin:

any progress on this awesome feature?

I'm assuming the systemd-boot signed options aren't available yet?

@shadowrylander they are if you use the PR branch.

@lheckemann Could you give me a code example? I'm still fairly new to NixOS, and having trouble understanding the syntax.

You could use the current state of the branch, but that comes with the disadvantage that it's over a year old now and you'll get over-a-year-old software (without security patches from that period) correspondingly. If you're willing to put the effort in to rebase it on current master (which will take at least a basic understanding of nix syntax so you can work out how to fix the conflicts), you can then build your system from it by passing -I nixpkgs=$path/to/your/nixpkgs to nixos-rebuild. Otherwise no, it's not available yet.

Hmm... Well, it should be a great way to learn the syntax, at the very least; I think I'll try it. Thanks!

Friendly ping

Anyone has the skills to pick up this PR?

I can add the code from this PR still works fine on 20.09 when used as a module.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nixos-on-secure-boot-debian-based-laptop/10976/2

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nixos-on-secure-boot-debian-based-laptop/10976/2

@danielfullmer Do you feel like taking this forward?  It seems that @grahamc doesn't have time to push this anymore.  I'm actually using this code for almost a year already and it would be really great if it made into 21.05.

@danielfullmer Do you feel like taking this forward?  It seems that @grahamc doesn't have time to push this anymore.  I'm actually using this code for almost a year already and it would be really great if it made into 21.05.

As an observer, what extra work is required to make this ready? (Excluding resolving merge conflicts)

As an observer, what extra work is required to make this ready? (Excluding resolving merge conflicts)

I think for something so security critical it needs an automatic VM test. I think that is pretty much it.
On Sat, Jan 16, 2021, at 1:22 PM, Jack Kelly wrote:

As an observer, what extra work is required to make this ready? (Excluding resolving merge conflicts)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/NixOS/nixpkgs/pull/53901#issuecomment-761610801, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCEZ2WQ7AHHSYX74VCXMTTS2HKPHANCNFSM4GPVW2YA.

I think for something so security critical it needs an automatic VM test. I think that is pretty much it.
On Sat, Jan 16, 2021, at 1:22 PM, Jack Kelly wrote:

As an observer, what extra work is required to make this ready? (Excluding resolving merge conflicts)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/NixOS/nixpkgs/pull/53901#issuecomment-761610801, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCEZ2WQ7AHHSYX74VCXMTTS2HKPHANCNFSM4GPVW2YA.

@hmenke I unfortunately can't commit to having time to finish up this PR.  I should be available to answer any questions about the tests if anyone else would like to finish this up.  I'd also like to add that we'd also need some documentation for this feature.
However, I do have a slightly more recent version of the secure boot tests at: https://github.com/danielfullmer/nixpkgs/tree/secure-boot-wip2
The biggest change is that the logic for adding secure boot certs was moved into qemu-vm.nix.  IIRC, nixos-rebuild build-vm-with-bootloader should now also be able to pre-enroll the certificate specified under boot.loader.systemd-boot.signing-certificate, so you could interactively test secure boot functionality.
I might not be the best to finish up this PR since I personally wasn't planning to use this feature on my machine until we also had working kernel module signing (https://github.com/NixOS/nixpkgs/pull/87426) and had chained integrity checking to the actual generation NixOS boots via dm-verity / fs-verity / IMA, or something similar, for which I don't have any great ideas.

@hmenke I unfortunately can't commit to having time to finish up this PR.  I should be available to answer any questions about the tests if anyone else would like to finish this up.  I'd also like to add that we'd also need some documentation for this feature.
However, I do have a slightly more recent version of the secure boot tests at: https://github.com/danielfullmer/nixpkgs/tree/secure-boot-wip2
The biggest change is that the logic for adding secure boot certs was moved into qemu-vm.nix.  IIRC, nixos-rebuild build-vm-with-bootloader should now also be able to pre-enroll the certificate specified under boot.loader.systemd-boot.signing-certificate, so you could interactively test secure boot functionality.
I might not be the best to finish up this PR since I personally wasn't planning to use this feature on my machine until we also had working kernel module signing (https://github.com/NixOS/nixpkgs/pull/87426) and had chained integrity checking to the actual generation NixOS boots via dm-verity / fs-verity / IMA, or something similar, for which I don't have any great ideas.

WOW! I kinda wanted to have this and now it exists.
I don't think the security would suffer too much from the fact that files are re-signed on every nixos-rebuild boot, if the files are re-installed from the /nix/store every time. Then, if the system booted successfully with Secure Boot from the beginning, and the /nix/store wasn't tampered with, I don't think there's a chance someone untrusted could swap the bootloader. Quite the contrary, every rebuild will ensure the system is in bootable state and all the files are trusted.

Thanks a lot for this module.
I've managed to merge it with nixos 21.05:
https://github.com/cwyc/nixpkgs/tree/secure-boot-rebased-21.05/nixos/modules/system/boot/loader/systemd-boot
And I've adapted it as an importable module, with the options renamed to boot.loader.systemd-boot-secure:
https://gist.github.com/cwyc/9de83566c92bdabd000256ed49312a53

@cwyc Are you going to open a PR? :pray:

Maybe open a new PR and ask for reviews again.

Please, please, if someone has the time and knowledge to take this over the finish line, please do. We will apparently not get very many testers without this having landed in nixpkgs.

@mweinelt I'm definitely interested in seeing this land - what's still missing from the changes that @cwyc posted above? Whilst as mentioned earlier without having chained verification of kernel modules etc we don't get full trust, this is definitely a significant stepping stone to get there.

@benpye I think a start would be addressing the review comments - there are plenty right there, some really low hanging fruit!

Please, please, if someone has the time and knowledge to take this over the finish line, please do. We will apparently not get very many testers without this having landed in nixpkgs, and the value of having this built-in and easily set up is so huge.

It's something that really ought to be merged and functional by the time of the next release, because a lot of people are about to lose the option of running their systems with secure boot disabled.
Given Windows 11 has it as a requirement, any operating system which doesn't support it at all can't boot. Not without reconfiguring your BIOS at every reboot, at any rate, which I don't think many people are going to do. Personally I've been running an indev version of 11, and...
Well, it's a good upgrade. With the exception of the usual development snapshot bugs, most everything works better than on 10.
I don't expect anyone who's already using NixOS to be stopped by this, so it might not seem that important, but losing the ability to dual-boot will make it far harder to make new converts.

lose the option of running their systems with secure boot disabled.

not without reconfiguring your BIOS at every reboot

This sounds like FUD to me. I don't see why Windows11 making it a requirement makes the hardware or firmware behave differently than it has for years.
I have a different approach to this PR in progress, but I don't think this particular implementation is good or worth merging: the lifecycle around signing is unfit for purpose. To that end, I'm closing this because I would be -1 on an implementation similar to this merging.

@Baughn
Looks like it's only an issue if you want to dual-boot with Windows 11. But if you are dual-booting, surely you can't replace Microsoft's keys with the self-signing keys used here, a secure boot solution would have to use shim/preloader.

@Baughn
surely you can't replace Microsoft's keys with the self-signing keys used here

You can import several signing keys in UEFI. This way, I have my own Platform Key, two Key Exchange Keys (my own and Microsoft's, so Windows can autoupdate dbx), two keys in db (again, my own and Microsoft's), and dbx is full of Microsoft's revoked signs. This way, I have Secure dual-boot with Windows 10 and Ubuntu (autosigned by https://github.com/julian-klode/sicherboot).
Installing NixOS requires disabling Secure Boot, which will somewhat lower my security, and this is one of the few reasons that stops me from daily driving NixOS.

Given Windows 11 has it as a requirement

that only seems to be true for installation. I installed win 11 then turned off secure boot and am now back to dual booting win 11 and nixos

@grahamc do please link your solution here once it's in a state worth looking at so we get a notification :)

I don't know if Windows 11 has it as a requirement but I have been using it since release to Dev and the like past 3 updates and have not gotten anything needing Secure Boot to be re-enabled (whether during upgrade process or daily driving).

The source would be https://blogs.windows.com/windows-insider/2021/06/28/update-on-windows-11-minimum-system-requirements/
It's true that at the moment you can disable secure boot after installation, and it'll keep running, but I don't think it's especially safe to assume that will still be the case with the final release. I've never seen any statement from MS saying it will be, and I don't think it'd be safe to assume even if I had.

Okay. Good to confirm there is no imminent risk here.

I have been meaning to implement an alternative to this myself for a while but I finally got round to it. This completely replaces the configuration file approach with unified kernel images.
https://github.com/frogamic/nix-machines/tree/main/modules/systemd-secure-boot
Some caveats:

signing is not optional in this script, though sign_if_not_signed could easily be replaced with different behavior if signing is not enabled, sticking with unified images by default for systemd-boot
doesn't sign memtest
doesn't sign fwupd bootloader as used for platform firmware updates
Leaves behind all the old initrd/bzimage files in EFI/Nixos, should the eventual script nuke these every time on the off-chance they exist?

Some thoughts I had while doing it:

Could the unified image be built as a nix derivation instead? Probably in the extraSystemBuildCmds since not all boot methods need it?

What happens in that case if a generation doesn't have the unified image? is it just skipped?

Should secureboot be a configuration option on the same level as systemd-boot rather than inside it so all boot methods could share it as an option, and any other efi binaries (eg fwupdx64.efi) would use it too?

can you even sign your own grub efi like you can with systemd-boot? I never looked into it at all

I've used @frogamic approach on my laptop and I'd really like to see this in NixOS.
Is anyone working on a new PR at the moment?
A few comments I have:

I'd say defaulting to unified images is not a great idea, since it takes a lot more space
maybe splitting this in two options would be better, the secure boot option requiring to enable the unified images option?
a nice to have would be an option to build efitools' LockDown.efi with the user's certificates and add it to the boot menu

Is anyone working on a new PR at the moment?

Yes, my company determinate.systems has been investing considerable time and money in to a suitable implementation over the past two months. We're about ready to publish some WIP code. Typically we do all our work in public, however I wanted to have lots of room to experiment and try different approaches. We're also in the process of applying for funding to implement a few workflows.


False alarm in back-to-back file generation

Describe the bug
When creating an output directory (e.g., in Verilog target generation) back-to-back, there would be an error message saying directory creating fails. However, the directory already exists and the file generation is Okay.
To Reproduce
Steps to reproduce the behavior:

Generate Verilog target
Generate Verilog target again -- with the same output path
See error

Environment (please complete the following information):

OS: Ubuntu 18.04
Compiler gcc 7.4.0

Additional context
Check status code to be -1 instead of 0.

Do you see an error or a warning?
Currently it should be something like this:
W0607 21:13:49.464702 10027 single_target.cc:1093] Cannot create output directory:../verification/XX

If this is what you see, I don't see why this is a bug. The user should be prompted that of such issues in case there are other potential problems (directory accessing privilege is ill-set).

I guess the "bug" means the user needs to handle the warning in his program but not let it be printed out.

I gave it a bug tag just because there's no better tag for this... A better solution is to make the warning message clearer -- differentiate already-exist from create-fail (which can be due to privilege issue).
No worry, I've changed that in a branch and will PR soon.


Automated cherry pick of #6849: fix: disable policy rule compaction

Cherry pick of #6849 on release/3.1.
#6849: fix: disable policy rule compaction

Codecov Report

Merging #6851 into release/3.1 will decrease coverage by 0.37%.
The diff coverage is 9.05%.

@@               Coverage Diff               @@
##           release/3.1    #6851      +/-   ##
===============================================
- Coverage         7.47%    7.10%   -0.38%
===============================================
  Files              585      600      +15
  Lines            98936   105871    +6935
===============================================
+ Hits              7396     7517     +121
- Misses           90967    97778    +6811
- Partials           573      576       +3

Flag
Coverage Δ

#aFlag
7.10% <9.05%> (-0.38%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/cloudcommon/policy/policy.go
1.57% <0.00%> (-0.01%)
:arrow_down:

pkg/cloudcommon/policy/resources.go
0.00% <ø> (ø)

pkg/cloudprovider/objectstore.go
2.51% <0.00%> (-0.01%)
:arrow_down:

pkg/cloudprovider/securitygroup.go
0.00% <0.00%> (ø)

pkg/compute/guestdrivers/managedvirtual.go
0.00% <0.00%> (ø)

pkg/compute/models/buckets.go
1.24% <0.00%> (ø)

pkg/compute/models/cloudproviders.go
0.96% <0.00%> (-0.01%)
:arrow_down:

pkg/compute/models/disks.go
0.55% <0.00%> (-0.01%)
:arrow_down:

pkg/compute/models/guestdisks.go
0.58% <ø> (+0.01%)
:arrow_up:

pkg/compute/models/guestnetworks.go
0.21% <0.00%> (+<0.01%)
:arrow_up:

... and 52 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update facbe54...120ed1f. Read the comment docs.


Use a Ledger instead of a bare UTXOSet

E_NEEDSTESTING
Fixes #150

So far found https://github.com/bosagora/infrastructure/issues/220 and https://github.com/bosagora/agora/pull/2881 and there seems to be another one, at least.
Pre-requisite : #165 .

Latest issue:
2022-01-19 14:34:23,281 Info [agora.consensus.state.Ledger] - Completed externalization of block #19
2022-01-19 14:34:23,305 Info [agora.consensus.state.Ledger] - Beginning externalization of block #20
2022-01-19 14:34:23,305 Info [agora.consensus.state.Ledger] - Transactions: 0 - Enrollments: 6
2022-01-19 14:34:23,305 Info [agora.consensus.state.Ledger] - Validators: Active: 20 - Signing:<PHONE_NUMBER>1111111111 - Slashed: 0
2022-01-19 14:34:23,306 Fatal [agora.consensus.state.Ledger] - Error while adding a new validator: This validator is already enrolled
2022-01-19 14:34:23,306 Fatal [agora.consensus.state.Ledger] - Enrollment #0: { utxo_key: 0x0666c4d505b55b6840fbb669ec08a1849e699d5a30ba246989b65ea71292f8ac9a3d7126ca9061313d3225d6e324146f37cdc5dab51facbbc3beead6854e89a4, commitment: 0xacc94967f8ad17f622c883c782cdcc01387e7e49d6c5cadaf3db6f4744895dfedefa115d15667bbc0904ec3f01358afbb6b05bf5980b2f47e59b2d9a01378062, enroll_sig: 0xf1d9d0a109830f568657023912709ff8e2bb2ca4b1e58ade48a52424e057e67607c05a89c14ef8e81b5800254f62249df7bfbf28881aeaedfa4db6ac1d141dfd }

This seems to be working. The issue I had was because my config file didn't have the correct ConsensusConfig, although it should not have crashed.


fix: broken link

Will fix https://github.com/yaooqinn/kyuubi/issues/515
Why are the changes needed?
To fix a broken image ref link
How was this patch tested?
Not necessary

thanks merging to master/1.1


How do I share dependencies between solutions

It is recommended to use manifest mode by default, so I am trying to play with it.
Suppose I develop a static library MyLib that uses Boost.  I define "boost" as its dependency in the manifest. Then I have a project MyProj that uses MyLib. I include its header like #include <MyLib/myheader.h> and immediately get a compilation error because myheader.h happens to use Boost, and of course MyProj has no idea about it. Furthermore, if I list boost as MyProj's dependency, it might end up getting a different version of Boost, which doesn't sound nice.
Thus, it seems that there is a need to be able to share manifests across solutions of I am doing something wrong. Would much appreciate any advice.

Is MyProj in the same solution as MyLib? We use a single manifest for all projects inside a solution. If you use vcpkg to depend on MyLib you can add boost as a dependency in the port files vcpkg.json

No, unfortunately. I know I can share the same manifest within the same sln, but the issue arises is I create a library that is later used in several other solutions.

MSBuild does not have any kind of external dependency lookup. Either you use vcpkg in the whole tree or you just start and use CMake which takes care of this for you.

Are the solutions in the same git? And how do you distribute MyLib so that you can use the lib in other solutions?

@Neumann-A, Right, but the question is what is "tree". In case of vcpkg it seemingly means "everything under the same sln file", which is quite restrictive.
@autoantwort, I am flexible in my setup, but currently what I do is this. I have some homemade libraries I use in several projects. I keep them under %MY_LIBS% path. When I want to use a library, I simply add `%MY_LIBS% to the include and library search paths of a project.


Remove deprecated update_docs

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Deprecation cleanup

What is the current behavior? (You can also link to an open issue here)
Add_or_update docs endpoint is still available

What is the new behavior (if this is a feature change)?
Removes add_or_update docs endpoint

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
Yes. If you are still using add_or_update docs, this will no longer work for you.

Have unit tests been run against this PR? (Has there also been any additional testing?)
in progress

Related Python client changes (link commit/PR here)
here

Related documentation changes (link commit/PR here)
n/a

Other information:

Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines
[x] Tests for the changes have been added (for bug fixes/features)
[ ] Docs have been added / updated (for bug fixes / features)

Tests passing:

arm64 API tests: https://github.com/marqo-ai/marqo/actions/runs/5385753028
unit tests: https://github.com/marqo-ai/marqo/actions/runs/5385751889

unit tests: https://github.com/marqo-ai/marqo/actions/runs/5401423357
CUDA DIND tests: https://github.com/marqo-ai/marqo/actions/runs/5401424359

unit tests: https://github.com/marqo-ai/marqo/actions/runs/5401445292
cuda API tests: https://github.com/marqo-ai/marqo/actions/runs/5401446726

Another note: You also may want to edit the tests in test_add_documents_use_existing_tensors.py, as they have update_mode set to replace as well

Updated these too, and removed other remaining references I found to update_mode


fix: supress mouse4 and mouse5 browser navigation which could break the app

The location of this snippet may be "controversial".
I chose it mainly because it's such a central adjustment and it doesn't really fit into any Vue file I feel.
Though I'm not opposed to changing the location.

I did test it manually and it does indeed prevent it the buttons from navigating.
I didn't choose the "app-command" alternative based on the fact that the issue within electron's repo is still open.

Closes #654

I agree that this might not be the most optimal place, but it works right now :)
If we end up with too much in this file, we can move it around somewhere else.
For now, looks good to me!
Thanks for the contribution :)


fix(bundle): sets IKE_VERSION for making bundle

Short description of what this resolves:
IKE_VERSION sets explicitly, as tag does not exist yet on the commit for which bundle should be made.
Changes proposed in this pull request:

release.sh updates

Closing as we convinced GH to 500 on that given PR. New one recreated as #676


Examples about how to calculate wave function at given position

Are there some examples about how to calculate psi (wave function) as given position (NElectron, 3)

Examples of dumping to chgcar or cubgen format
https://github.com/pyscf/pyscf/blob/master/examples/tools/05-cubegen.py#L24
https://github.com/pyscf/pyscf/blob/master/examples/tools/06-chgcar.py#L21
You can trace to the code implementation if you need other format. The implementation is straightforward

Values of orbitals at given postion can be evaluated
xyz = np.random.rand((100, 3))
np.einsum('pi,rp->ri', orbtial_coef, mol.eval_gto('GTOval', xyz))

There is no function to evaluate N-electron wavefunction
On Thu, Dec 10, 2020 at 04:30:58AM -0800, Xiang Liang wrote:

Are there some examples about how to calculate psi (wave function) as given position (NElectron, 3)
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/pyscf/pyscf/issues/780

--


Regarding Terraform integrated with jenkins (need docs)

Please provide docs or step for how can i integrated my tarraform (.tf) file with Jenkins.

https://dzone.com/articles/immutable-infrastructure-cicd-using-hashicorp-terr

Terraform Pipelines in Jenkins:
https://medium.com/@timhberry/terraform-pipelines-in-jenkins-47267129ff06
Integrate Terraform with Jenkins using plugin
:https://medium.com/@khandelwal12nidhi/integrate-terraform-with-jenkins-d283dd5d819c

Hi
Thanks for your docs, i followed he docs and create a jenkins pipeline
pipeline {
agent any
stages {
stage(‘checkout’) {
steps {
git branch: 'master', credentialsId: 'XXXXXXXXXXXXX', url: 'https://github.com/khatrigopal/terraform-jenkins.git'
    }
}

// stage('Set Terraform path') {
// steps {
// script {
//  def tfHome = tool name: ‘Terraform’
// env.PATH = "${tfHome}:${env.PATH}"
//}
// sh 'terraform — version'
 //}

// }
stage('Provision infrastructure') {

    steps {
         dir('myfirstec2')
         {
         sh 'terraform init'
         sh 'terraform plan -var "accesskey="${accesskey}" -var "secretkey="${secretkey}" -out=plan'
         // sh 'terraform destroy -auto-approve'
         sh 'terraform apply plan'
         }

    }

}
}
}
and create 2 parameter for access key and secret key.

but my question is i m providing hard core value for access key and secret key in parameter but i dont want it.
Please resolve this issue.

you can use credentials to know more about it refer here

Hi
How can we use these credentials plugin for ACCESSKEY and SECRETKEY.


HDDS-2834. Directly read into ByteBuffer if it has array

What changes were proposed in this pull request?

Use target ByteBuffer's array, if it has one, to avoid buffer copy in OzoneFSInputStream.
Skip ByteBuffer#put on EOF (see similar logic in CryptoInputStream).
Rename TestOzoneFSInputStreamUnit to TestOzoneFSInputStream since integration test with same name was moved out to integration-test.  Also move it to the correct package (where OzoneFSInputStream resides).

https://issues.apache.org/jira/browse/HDDS-2834
How was this patch tested?
Tweaked unit test to exercise the read method with both heap and direct buffers.
https://github.com/adoroszlai/hadoop-ozone/runs/377273986

@fapifta please review if you have some time

Thank You @adoroszlai for the contribution.

Thanks @bharatviswa504 for reviewing and merging this.


[Bug]: json bucket manifest typo for openscad-dev

Prerequisites

[X] I have written a descriptive issue title.
[X] I have searched all issues/PRs to ensure it has not already been reported or fixed.
[X] I have verified that I am using the latest version of Scoop and corresponding bucket.

Package Name
openscad-dev
Expected/Current Behaviour
openscad-dev package fails to install after extraction. Appears to be a typo in the extract_dir property
Steps to Reproduce
PS> scoop install openscad-dev
Installing 'openscad-dev' (2023.03) [64bit] from versions bucket
OpenSCAD-2023.03.02-x86-64.zip (50.5 MB) [====================================================================] 100%
Checking hash of OpenSCAD-2023.03.02-x86-64.zip ... ok.
Extracting OpenSCAD-2023.03.02-x86-64.zip ... Could not find 'openscad-2023.03.02'! (error 16)
At C:\Users\xxxxx\scoop\apps\scoop\current\lib\core.ps1:638 char:9
+         throw "Could not find '$(fname $from)'! (error $($proc.ExitCo ...
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Could not find ...02'! (error 16):String) [], RuntimeException
    + FullyQualifiedErrorId : Could not find 'openscad-2023.03.02'! (error 16)

Possible Solution
Change "extract_dir": "openscad-2023.03.02" to "extract_dir": "openscad-2023.03.02-x86-64" and "extract_dir": "openscad-2023.03.02-x86-32"
Scoop and Buckets Version
PS> scoop --version
Current Scoop version:
v0.3.1 - Released at 2022-11-15

'extras' bucket:
4ee5dbd3a (HEAD -> master, origin/master, origin/HEAD) todolist: Update to version 8

'main' bucket:
4baed96de (HEAD -> master, origin/master, origin/HEAD) usql: Update to version 0.14.0

'versions' bucket:
b4ec36b22 (HEAD -> master, origin/master, origin/HEAD) prismlauncher-qt5-git: Update to version 7.0-bc1a7d2

Scoop Config
PS> scoop config

last_update                       scoop_branch scoop_repo
-----------                       ------------ ----------
2023-04-02T15:20:06.0146494+02:00 master       https://github.com/ScoopInstaller/Scoop

PowerShell Version
PS> $PSversionTable

Name                           Value
----                           -----
PSVersion                      5.1.22000.832
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22000.832
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

Additional Softwares
No response

I've opened a PR to address this issue: https://github.com/ScoopInstaller/Versions/pull/1160

Bumping this issue, we have the PR ready and it need to be approved.

I can't install this package without this fix:
Installing 'openscad-dev' (2023.09) [64bit] from versions bucket
Loading OpenSCAD-2023.09.06-x86-64.zip from cache
Checking hash of OpenSCAD-2023.09.06-x86-64.zip ... ok.
Extracting OpenSCAD-2023.09.06-x86-64.zip ... Exception: C:\Users\User\scoop\apps\scoop\current\lib\core.ps1:638
Line |
 638 |          throw "Could not find '$(fname $from)'! (error $($proc.ExitCo …
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Could not find 'openscad-2023.09.06'! (error 16)


Improve the notification system for new firmware and IDE releases.

The "out-of-date" bar in the bottom of the IDE isn't enough and can't be seen on smaller screens. Move the notifications to the side bar and make them more visible.

Will make a bar that goes all the way across the bottom of the IDE using the QtCreator API for these bars.


Make user management page responsive

Describe the bug
Although it's not broken, the user management page has horizontal scrolling
To Reproduce

Go to the user management page on mobile (or a browser with a width < 768px)

Expected behavior
It's nicely formatted, and you don't have to horizontally scroll
Screenshots

Desktop

OS: Android 13
Browser: Firefox
Version: 116.2.0

Additional context
Maybe style like the Overview page on mobile?

Closed in 1.7.1


how i get registrationID oauth2 client, i have spring-boot 1.4.2 i try implementing oauth2 in spring-boot 1.4.2

Summary

Actual Behavior

Expected Behavior

Configuration

Version

Sample

i am using spring boot 1.4.2 and i am having a problem with oauth2 implementation,
how can i get google or facebook registration ID

Thanks for getting in touch, but it feels like this is a question that would be better suited to Stack Overflow. We prefer to use GitHub issues only for bugs and enhancements.


No?

Interesting.  Most product comparison sheets have "Yes" and "No".  But not here.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bbe4ef3a-6282-5c51-5810-c6d63f3a0e1b
Version Independent ID: 6d655e24-fc84-aad0-2422-2451cfb1ad34
Content: What is Azure Data Studio - Azure Data Studio
Content Source: docs/azure-data-studio/what-is.md
Product: sql
Technology: azure-data-studio
GitHub Login: @markingmyname
Microsoft Alias: maghan

@jeffshumphreys -- Jeff, thank you for your observation.  We are conserving ink  :^)
I am closing this issue now.  You are welcome to @ mention me for any followup, and we hope to hear from you again.


SciBert Checkpoints not compatible with Bert

As written in the github repo of scibert ,we can use the scibert checkpoints for Tensorflow by plugging them in the Bert code on google-research/bert ( https://github.com/google-research/bert ) but when i tried doing the same I got the following data loss error -
DataLossError: Unable to open table file /content/scibert_data/scibert_scivocab_uncased/bert_model.ckpt.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

How are you plugging them into the BERT code? do you a code snippet or command line?

Looks like you are using
--init_checkpoint=scibert_scivocab_uncased/bert_model.ckpt.data-00000-of-00001
but you should use the following instead
--init_checkpoint=scibert_scivocab_uncased/bert_model.ckpt  (even though bert_model.ckpt is not actually a file. This is tf thing).

I can concur that it's a simple swap to get this running in place of BERT-large.

Sorry one parameter to tweak that I forgot -pool_layer=-1 rather than the default which is -2 that gave much better results at least with initial testing.
Aand to be clear treat scibert as a pretrained model rather than a fine-tuning.


VRX Crashes Every Couple Minutes

No matter which launch file I use, VRX crashes after a couple minutes and I have to restart everything. It is really starting to affect my productivity. These are the commands I'm running:
roslaunch vrx_gazebo scan_and_dock.launch urdf:=$HOME/my_wamv/my_wamv.urdf
roslaunch wamv_gazebo localization_example.launch
roslaunch wamv_gazebo rviz_vrx.launch
Every several minutes, the first terminal (vrx) errors out with:
Segmentation fault (core dumped)
[gazebo-2] process has died [pid 47690, exit code 139, cmd /opt/ros/noetic/lib/gazebo_ros/gzserver -e ode /home/robotx/vrx_ws/devel/share/vrx_gazebo/worlds/dock.world __name:=gazebo __log:=/home/robotx/.ros/log/64c56d8e-1fa9-11ec-9a4e-c99f73aa7815/gazebo-2.log].
log file: /home/robotx/.ros/log/64c56d8e-1fa9-11ec-9a4e-c99f73aa7815/gazebo-2*.log
I would love to know if it is user error or if it is something that needs to be fixed.
Many thanks.

Hi @ayarst , this is probably expected because you're running a world with a scoring plugin that is going to terminate the world after 5 mins of running mode. I recommend running vrx.launch or modify the <running_state_duration>300</running_state_duration> in the world file.
By the way, scan_and_dock is from the 2019 competition. The 2022 world is scan_dock_deliver.

@caguero thank you so much. I will look into that immediately! Really appreciate the quick response time.

@caguero while I have you, I actually have one other question. Where exactly is wamv/base_link located? I am trying to set up my sensors_config.yaml and without knowing the location of base_link relative to our robot frame, I have to eyeball it and I do not want to do that.

Can't you see the wamv/base_link frame link in rviz? I'm not sure if I can give you a precise location as this is the main origin of the model.

Yeah I can, was just wondering if there was something more you could give me. No worries, I'll figure it out. I really appreciate your help.


changed "Linking FFmpeg..." to "Links FFmpeg..." in the readme

Made a minor change to the readme that felt contextually correct.

Thanks!


ARROW-16918: [Gandiva][C++] Adding UTC-local timezone conversion functions

Adding functions
to_utc_timezone : Converts a timestamp from local timezone to UTC time
from_utc_timezone : Converts a timestamp from UTC time to local time

We can fix style by cmake --build ${BUILD_DIR} --target format but you need to install clang-format-12.
If it's difficult to prepare clang-format-12, I can fix style and push the fix to this branch.


Implement primes

Fix #113
Also slightly change vertical position of subscripts.

Codecov Report
Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (c8cb679) 79.49% compared to head (5a27bfd) 74.04%.

Files
Patch %
Lines

src/engine/layout.jl
50.00%
5 Missing :warning:

src/parser/parser.jl
80.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #115      +/-   ##
==========================================
- Coverage   79.49%   74.04%   -5.46%
==========================================
  Files          10        7       -3
  Lines         556      470      -86
==========================================
- Hits          442      348      -94
- Misses        114      122       +8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Surface Laptop 3] Keyboard and touchpad don't seem to work

I've run the latest version (d3318d2c) of the setup.sh and compile_setup.sh with kernel 5.3, installed it -- the scripts worked fine** and I was able to boot on it but unfortunately touchpad and keyboard still don't seem to work. Have I missed anything? Is SL3 supposed to be supported already?
** I just had errors in the setup.sh with the following modules not found, but the module for keyboard seem to be found fine
==> ERROR: module not found: `intel_ipts'
==> ERROR: module not found: `surface_sam_ssh'
==> ERROR: module not found: `surface_sam_san'
==> ERROR: module not found: `surface_sam_vhf'

What's the output of uname -v?

@dmhacker it says 5.3.15-1; not sure why there is no surface as I booted on the linux linux-surface kernel from grub. I genereted the grub config grub-mkconfig -o /boot/grub/grub.cfg and the entry Arch Linux, with Linux linux-surface points to /boot/vmlinux-linux-surface which seems correct (?)
I've noticed you made new updates so I tried to re-compile and install, now uname -r  shows 5.18.10-1-surface but the keyboard still doesnt work.

Can you try updating to the 5.3.18 kernel? Have you installed all firmware and the libwacom-surface package?

My bad, I made a typo above, i meant to say I've installed and tried kernel 5.3.18-1-surface  (and checked with uname -r that it's running the right kernel). I've installed all firmware through the setup.sh script -- answered y to all and installed libwacom-surface from https://aur.archlinux.org/libwacom-surface.git , unfortunately that did not solve the keyboard issue.

I'm not sure why the keyboard isn't working then. It could be that the kernel doesn't support the SL3 yet. Maybe @qzed can help.

The kernel should support the SL3 keyboard, I wrote the patch for it. Make sure to run the 5.4 version of the kernel though:

5.3 contains a reverted i915 driver which can cause issues on the newer devices (https://github.com/linux-surface/linux-surface/issues/23)
5.4 contains a handful of Ice Lake specific stuff that makes things like the power button work
5.4 lacks the ipts driver which breaks touchscreen, but the newer devices use a different protocol so it won't work with the 5.3 driver anyway

Thanks both! I had a look at the compile_setup.sh script, looks like I'm only missing the file base/configs/config5.4 to install the kernel using @dmhacker 's script, any idea how I can generate this config?

You can find that file here, just wget the raw version as thebase/config/config.5.4 file you mentioned earlier. I'll add this to the repository later tonight.

Thanks -- for some reason when re-building it and installing, keyboard still didn't work, but after installing  v5.4.6-1 directly from https://github.com/linux-surface/linux-surface/releases it worked fine. So I was probably doing something wrong :/
Btw, @dmhacker in order to add 5.4 to the compile_setup.sh, it looks like you're missing it in the options line 8
Closing the issue :)

It could be that I gave you the wrong config actually, since that's the base config for the unpatched 5.4.1 kernel. Using this one might give you better results. I've updated the repository with it.
Regardless, glad you got it working!


Update FrontController.php. Issue with ordering.

Since 5.7 order by in sub query are not supported. This modification tells Fanta pager to use simple query handling.

Good catch, thanks @Ninored.


FROM_EMPTY_HASH-> FROM_EMPTY_64BIT_HASH

It looks like FROM_EMPTY_HASH in lib/reader/marc.py should be changed to FROM_EMPTY_64BIT_HASH.

Correct. Hmm. Looks as if I'd only checked in half of that update. Fixed.


android app is crashing after simply installing the library

I have latest gradle version in my react native app

upgrading npm verision to the latest solved my problem.


Animate promises not firing from directive inside a view

I've created a custom directive that fires $animate.enter when its created. Works fine until I place the directive within a view and then it only works sporadically. The only way I can get it work is to use a very long timeout or change the parent container to the body. Why?
This is the directive and animation
app.directive('ngCustom',  function ($animate, $timeout, $rootScope) {

  return function(scope, element, attrs) {
    var parent = element.parent();//angular.element(document.body);
    $timeout(function(){
      $animate.enter(element, parent);
    });
  }
});

app.animation('.animate-custom', [ function () {
    var obj = {};
    obj.enter = function(element){
      TweenMax.from(element.find("h1"), 0.5, {delay:1, opacity:0, x:"+=100"});
    }
    return obj;
}]);

and a sample app with the directive outside and inside a view
http://plnkr.co/edit/HEDUIAifPEB6TbKNdm48?p=preview
I've also tried the addClass method and have a similar issue
http://plnkr.co/edit/VIGYpOfu6QIknZJBKEKi?p=preview

Add ng-animate-children to <div ng-view>.
http://plnkr.co/edit/nxu3bF9q1b0gthKNkH7d?p=preview

There will be a much better API for this in 1.4.

sweet, thanks for that. Any docs 1.4 animate api yet?

@fatlinesofcode there's a giant refactor going on for 1.4. This refactor will open up more possibilities with animate and it will make parent/child parallel and blocked animations much easier to do. So far it's in the prototypical stage.

@matsko Looking forward to it, any tie in with whats happening with angularjs-gsTimelines ? https://github.com/ThomasBurleson/angularjs-gsTimelines

@fatlinesofcode there is a 100% tie in with that, but the outcome of it will be more automated with how angular works and the DSL will change a bit.


Add a note about making a non-technical thread for critical support issues

Example: Technical thread - Non-technical thread
What do you think @sweir27 ?

I like this, one thread can be noisy: everyone's getting notifications, engineers are possibly using it for dev communications (although sometimes that goes off into a more relevant channel to be worked on), and stakeholders want updates.
Great idea!

This sentiment is spot on-- it's been hard to separate setting expectations for stakeholders from actual debugging.
I'm not sure if the short or long-term solution is a separate thread or something more "meaty".
Maybe @joeyAghion and I can explore this on Friday during our "Jira Ops" future friday experiment (and follow-up here!).

(:ceiling-cat: Any follow-up?)

@joeyAghion and I have a follow-up for the Jira ops exploration that we did-- maybe we can aim to get that out early this week. I think it would remove the need for this.

Sounds exciting!

Yep


adding tuple to StLb instance results in incorrect whole_stones and remainder_lbs values

The remainder_lbs attribute of a StLb instance should never be more than 13, because at 14 it should increment whole_stones. Updating the values via the overloaded addition operator can result in such a case  occurring:
>>> spam = StLb([1, 0])
>>> spam += (0, 15)
>>> spam
StLb object: 1 stones and 15lbs [29lbs]
# Should be 2 stones and 1 lb

This is a bug that needs to be fixed, and we need tests to ensure that remainder_lbs is never more than 13, and that whole_stones ticks up appropriately.

This bug only affects cases where instance =+ (whole_stones, remainder_lbs). It doesn't happen in cases of instance + other instance.