#1272 Adding AWS ecr-viewer and s3 module infra

PULL REQUEST
Summary
What changes are being proposed?
Below is the terraform plan to add ecr-viewer service and s3 bucket policies for ticket #1272.
Terraform will perform the following actions:

  # module.cloudfront.aws_cloudfront_distribution.alb will be updated in-place
  ~ resource "aws_cloudfront_distribution" "alb" {
        id                             = "E32K7H0CRMQKDT"
        tags                           = {
            "Name" = "eks-alb-cloudfront"
        }
        # (20 unchanged attributes hidden)

      - origin {
          - connection_attempts = 3 -> null
          - connection_timeout  = 10 -> null
          - domain_name         = "k8s-phdiplaygrounddev-1f5c307cc0-429276606.us-east-1.elb.amazonaws.com" -> null
          - origin_id           = "eks-alb" -> null

          - custom_origin_config {
              - http_port                = 80 -> null
              - https_port               = 443 -> null
              - origin_keepalive_timeout = 5 -> null
              - origin_protocol_policy   = "http-only" -> null
              - origin_read_timeout      = 30 -> null
              - origin_ssl_protocols     = [
                  - "TLSv1.2",
                ] -> null
            }
        }
      + origin {
          + connection_attempts = 3
          + connection_timeout  = 10
          + domain_name         = (known after apply)
          + origin_id           = "eks-alb"

          + custom_origin_config {
              + http_port                = 80
              + https_port               = 443
              + origin_keepalive_timeout = 5
              + origin_protocol_policy   = "http-only"
              + origin_read_timeout      = 30
              + origin_ssl_protocols     = [
                  + "TLSv1.2",
                ]
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.eks.data.kubernetes_ingress_v1.ingress will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "kubernetes_ingress_v1" "ingress" {
      + id     = (known after apply)
      + spec   = (known after apply)
      + status = (known after apply)

      + metadata {
          + annotations      = {
              + "alb.ingress.kubernetes.io/group.name" = "phdi-playground-dev"
            }
          + generation       = (known after apply)
          + name             = "ingress"
          + resource_version = (known after apply)
          + uid              = (known after apply)
        }
    }

  # module.eks.helm_release.building_blocks["ecr-viewer"] will be created
  + resource "helm_release" "building_blocks" {
      + atomic                     = false
      + chart                      = "ecr-viewer-chart"
      + cleanup_on_fail            = true
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = true
      + id                         = (known after apply)
      + lint                       = false
      + manifest                   = (known after apply)
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "phdi-playground-dev-ecr-viewer"
      + namespace                  = "default"
      + pass_credentials           = false
      + recreate_pods              = true
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://cdcgov.github.io/phdi-charts/"
      + reset_values               = false
      + reuse_values               = false
      + skip_crds                  = false
      + status                     = "deployed"
      + timeout                    = 300
      + verify                     = false
      + version                    = (known after apply)
      + wait                       = true
      + wait_for_jobs              = false

      + set {
          + name  = "ecrViewerUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ecr-viewer"
        }
      + set {
          + name  = "fhirConverterUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/fhir-converter"
        }
      + set {
          + name  = "image.tag"
          + value = "v1.1.11"
        }
      + set {
          + name  = "ingestionUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ingestion"
        }
      + set {
          + name  = "messageParserUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/message-parser"
        }
      + set {
          + name  = "smartyAuthId"
          + value = "98681a8c-89f6-f5b9-f14d-c832ce28e9b4"
        }
      + set {
          + name  = "smartyToken"
          + value = "AB5Bd7y9KGgxghGrD635"
        }
      + set {
          + name  = "validationUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/validation"
        }
    }

  # module.eks.helm_release.building_blocks["fhir-converter"] will be updated in-place
  ~ resource "helm_release" "building_blocks" {
        id                         = "phdi-playground-dev-fhir-converter"
      ~ metadata                   = [
          - {
              - app_version = "1.1.1"
              - chart       = "fhir-converter-chart"
              - name        = "phdi-playground-dev-fhir-converter"
              - namespace   = "default"
              - revision    = 2
              - values      = jsonencode(
                    {
                      - fhirConverterUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/fhir-converter"
                      - image            = {
                          - tag = "v1.1.12"
                        }
                      - ingestionUrl     = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ingestion"
                      - messageParserUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/message-parser"
                      - smartyAuthId     = "98681a8c-89f6-f5b9-f14d-c832ce28e9b4"
                      - smartyToken      = "AB5Bd7y9KGgxghGrD635"
                      - validationUrl    = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/validation"
                    }
                )
              - version     = "0.1.7"
            },
        ] -> (known after apply)
        name                       = "phdi-playground-dev-fhir-converter"
        # (26 unchanged attributes hidden)

      - set {
          - name  = "image.tag" -> null
          - value = "v1.1.12" -> null
        }
      + set {
          + name  = "ecrViewerUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ecr-viewer"
        }
      + set {
          + name  = "image.tag"
          + value = "v1.1.11"
        }

        # (6 unchanged blocks hidden)
    }

  # module.eks.helm_release.building_blocks["ingestion"] will be updated in-place
  ~ resource "helm_release" "building_blocks" {
        id                         = "phdi-playground-dev-ingestion"
      ~ metadata                   = [
          - {
              - app_version = "1.1.1"
              - chart       = "ingestion-chart"
              - name        = "phdi-playground-dev-ingestion"
              - namespace   = "default"
              - revision    = 2
              - values      = jsonencode(
                    {
                      - fhirConverterUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/fhir-converter"
                      - image            = {
                          - tag = "v1.1.12"
                        }
                      - ingestionUrl     = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ingestion"
                      - messageParserUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/message-parser"
                      - smartyAuthId     = "98681a8c-89f6-f5b9-f14d-c832ce28e9b4"
                      - smartyToken      = "AB5Bd7y9KGgxghGrD635"
                      - validationUrl    = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/validation"
                    }
                )
              - version     = "0.1.7"
            },
        ] -> (known after apply)
        name                       = "phdi-playground-dev-ingestion"
        # (26 unchanged attributes hidden)

      - set {
          - name  = "image.tag" -> null
          - value = "v1.1.12" -> null
        }
      + set {
          + name  = "ecrViewerUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ecr-viewer"
        }
      + set {
          + name  = "image.tag"
          + value = "v1.1.11"
        }

        # (6 unchanged blocks hidden)
    }

  # module.eks.helm_release.building_blocks["message-parser"] will be updated in-place
  ~ resource "helm_release" "building_blocks" {
        id                         = "phdi-playground-dev-message-parser"
      ~ metadata                   = [
          - {
              - app_version = "1.1.1"
              - chart       = "message-parser-chart"
              - name        = "phdi-playground-dev-message-parser"
              - namespace   = "default"
              - revision    = 2
              - values      = jsonencode(
                    {
                      - fhirConverterUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/fhir-converter"
                      - image            = {
                          - tag = "v1.1.12"
                        }
                      - ingestionUrl     = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ingestion"
                      - messageParserUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/message-parser"
                      - smartyAuthId     = "98681a8c-89f6-f5b9-f14d-c832ce28e9b4"
                      - smartyToken      = "AB5Bd7y9KGgxghGrD635"
                      - validationUrl    = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/validation"
                    }
                )
              - version     = "0.1.7"
            },
        ] -> (known after apply)
        name                       = "phdi-playground-dev-message-parser"
        # (26 unchanged attributes hidden)

      - set {
          - name  = "image.tag" -> null
          - value = "v1.1.12" -> null
        }
      + set {
          + name  = "ecrViewerUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ecr-viewer"
        }
      + set {
          + name  = "image.tag"
          + value = "v1.1.11"
        }

        # (6 unchanged blocks hidden)
    }

  # module.eks.helm_release.building_blocks["orchestration"] will be updated in-place
  ~ resource "helm_release" "building_blocks" {
        id                         = "phdi-playground-dev-orchestration"
      ~ metadata                   = [
          - {
              - app_version = "1.1.1"
              - chart       = "orchestration"
              - name        = "phdi-playground-dev-orchestration"
              - namespace   = "default"
              - revision    = 2
              - values      = jsonencode(
                    {
                      - fhirConverterUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/fhir-converter"
                      - image            = {
                          - tag = "v1.1.12"
                        }
                      - ingestionUrl     = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ingestion"
                      - messageParserUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/message-parser"
                      - smartyAuthId     = "98681a8c-89f6-f5b9-f14d-c832ce28e9b4"
                      - smartyToken      = "AB5Bd7y9KGgxghGrD635"
                      - validationUrl    = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/validation"
                    }
                )
              - version     = "0.1.9"
            },
        ] -> (known after apply)
        name                       = "phdi-playground-dev-orchestration"
        # (26 unchanged attributes hidden)

      - set {
          - name  = "image.tag" -> null
          - value = "v1.1.12" -> null
        }
      + set {
          + name  = "ecrViewerUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ecr-viewer"
        }
      + set {
          + name  = "image.tag"
          + value = "v1.1.11"
        }

        # (6 unchanged blocks hidden)
    }

  # module.eks.helm_release.building_blocks["validation"] will be updated in-place
  ~ resource "helm_release" "building_blocks" {
        id                         = "phdi-playground-dev-validation"
      ~ metadata                   = [
          - {
              - app_version = "v1.1.1"
              - chart       = "validation-chart"
              - name        = "phdi-playground-dev-validation"
              - namespace   = "default"
              - revision    = 2
              - values      = jsonencode(
                    {
                      - fhirConverterUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/fhir-converter"
                      - image            = {
                          - tag = "v1.1.12"
                        }
                      - ingestionUrl     = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ingestion"
                      - messageParserUrl = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/message-parser"
                      - smartyAuthId     = "98681a8c-89f6-f5b9-f14d-c832ce28e9b4"
                      - smartyToken      = "AB5Bd7y9KGgxghGrD635"
                      - validationUrl    = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/validation"
                    }
                )
              - version     = "0.1.7"
            },
        ] -> (known after apply)
        name                       = "phdi-playground-dev-validation"
        # (26 unchanged attributes hidden)

      - set {
          - name  = "image.tag" -> null
          - value = "v1.1.12" -> null
        }
      + set {
          + name  = "ecrViewerUrl"
          + value = "https://k8s-phdiplaygrounddev.us-east-1.elb.amazonaws.com/ecr-viewer"
        }
      + set {
          + name  = "image.tag"
          + value = "v1.1.11"
        }

        # (6 unchanged blocks hidden)
    }

  # module.eks.kubectl_manifest.ingress will be updated in-place
  ~ resource "kubectl_manifest" "ingress" {
        id                      = "/apis/networking.k8s.io/v1/namespaces/default/ingresses/ingress"
        name                    = "ingress"
      ~ yaml_body               = (sensitive value)
      ~ yaml_body_parsed        = <<-EOT
            apiVersion: networking.k8s.io/v1
            kind: Ingress
            metadata:
              annotations:
                alb.ingress.kubernetes.io/group.name: phdi-playground-dev
                alb.ingress.kubernetes.io/scheme: internet-facing
                alb.ingress.kubernetes.io/target-type: ip
              name: ingress
            spec:
              ingressClassName: alb
              rules:
              - http:
                  paths:
                  - backend:
                      service:
                        name: phdi-playground-dev-ingestion-ingestion-service
                        port:
                          number: 8080
                    path: /ingestion
                    pathType: Prefix
                  - backend:
                      service:
                        name: phdi-playground-dev-fhir-converter-fhir-converter-service
                        port:
                          number: 8080
                    path: /fhir-converter
                    pathType: Prefix
                  - backend:
                      service:
                        name: phdi-playground-dev-message-parser-message-parser-service
                        port:
                          number: 8080
                    path: /message-parser
                    pathType: Prefix
                  - backend:
                      service:
                        name: phdi-playground-dev-validation-validation-service
                        port:
                          number: 80
                    path: /validation
                    pathType: Prefix
                  - backend:
                      service:
                        name: phdi-playground-dev-orchestration-orchestration-service
                        port:
                          number: 8080
                    path: /orchestration
                    pathType: Prefix
          +       - backend:
          +           service:
          +             name: phdi-playground-dev-ecr-viewer-ecr-viewer-service
          +             port:
          +               number: 8080
          +         path: /ecr-viewer
          +         pathType: Prefix
        EOT
        # (12 unchanged attributes hidden)
    }

  # module.s3.data.aws_iam_policy_document.ecr_viewer_s3_policy will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "ecr_viewer_s3_policy" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "s3:GetObject",
              + "s3:GetObjectAcl",
              + "s3:PostObject",
              + "s3:PostObjectAcl",
              + "s3:PutObject",
              + "s3:PutObjectAcl",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
            ]

          + principals {
              + identifiers = [
                  + "ecrViewerPutPostGetS3Policy",
                ]
              + type        = "AWS"
            }
        }
    }

  # module.s3.data.aws_iam_policy_document.orchestration_s3_policy will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "orchestration_s3_policy" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "s3:PostObject",
              + "s3:PostObjectAcl",
              + "s3:PutObject",
              + "s3:PutObjectAcl",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
            ]

          + principals {
              + identifiers = [
                  + "orchestrationPutAndPostS3Policy",
                ]
              + type        = "AWS"
            }
        }
    }

  # module.s3.aws_s3_bucket.s3 will be created
  + resource "aws_s3_bucket" "s3" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "phdi-playground-tfstate-dev"
      + bucket_domain_name          = (known after apply)
      + bucket_prefix               = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = {
          + "Environment" = "dev"
          + "Owner"       = "Skylight"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
    }

  # module.s3.aws_s3_bucket_policy.allow_s3_access_for_ecr_viewer will be created
  + resource "aws_s3_bucket_policy" "allow_s3_access_for_ecr_viewer" {
      + bucket = (known after apply)
      + id     = (known after apply)
      + policy = (known after apply)
    }

  # module.s3.aws_s3_bucket_policy.allow_s3_access_for_orchestration will be created
  + resource "aws_s3_bucket_policy" "allow_s3_access_for_orchestration" {
      + bucket = (known after apply)
      + id     = (known after apply)
      + policy = (known after apply)
    }

  # module.eks.module.eks_blueprints_addons.aws_eks_addon.this["kube-proxy"] will be updated in-place
  ~ resource "aws_eks_addon" "this" {
      ~ addon_version               = "v1.29.0-eksbuild.3" -> "v1.29.1-eksbuild.2"
        id                          = "phdi-playground-dev:kube-proxy"
        tags                        = {
            "Blueprint"  = "phdi-playground-dev"
            "GithubRepo" = "github.com/aws-ia/terraform-aws-eks-blueprints"
        }
        # (9 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.eks.module.eks_blueprints_addons.aws_eks_addon.this["vpc-cni"] will be updated in-place
  ~ resource "aws_eks_addon" "this" {
      ~ addon_version               = "v1.16.2-eksbuild.1" -> "v1.16.3-eksbuild.2"
        id                          = "phdi-playground-dev:vpc-cni"
        tags                        = {
            "Blueprint"  = "phdi-playground-dev"
            "GithubRepo" = "github.com/aws-ia/terraform-aws-eks-blueprints"
        }
        # (9 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.eks.module.eks_blueprints_addons.module.karpenter.helm_release.this[0] will be updated in-place
  ~ resource "helm_release" "this" {
        id                         = "karpenter"
        name                       = "karpenter"
      ~ repository_password        = (sensitive value)
        # (30 unchanged attributes hidden)

        # (8 unchanged blocks hidden)
    }

Plan: 4 to add, 10 to change, 0 to destroy.

Additional Information
Anything else the review team should know?

terraform/implementation/variables.tf was added after I removed an extra space to the code.  This happened after I originally added my AWS TF changes in the Azure implementation folder instead of the AWS implementation folder.  The removal of the extra space will not change the Azure code as indicated by the plan.

#1272 Adding AWS ecr-viewer and s3 module infra

#53 Adding ecr-viewer and s3 module infra

@KennethSkylight, great point! The variables were added to Github.


Clean up build workflows

This repository was created some months before reusable workflows became generally available with the intention to use those later.
That hasn't happened yet, and as such, the workflows still have largely duplicate code.

Now addressed in https://github.com/bioconda/bioconda-utils/pull/953 via a unified build script.


error 500 on creating a repo, about repository_pkey not unique after update

Gitea version (or commit ref):1.13.2
Git version:2.25.1
Operating system:Ubuntu 20.04
Binary downloaded from gitea and replaced the old version(stopping gitea before this)

I'm running Gitea with systemd and it's gitea.service

Database (use [x]):

[x] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[x] No

Log gist:

2021/02/15 18:23:43 ...m.io/xorm/core/db.go:286:afterProcess() [I] [SQL] INSERT INTO "repository" ("owner_id","owner_name","lower_name","name","description","website","original_service_type","original_url","default_branch","num_watches","num_stars","num_forks","num_issues","num_closed_issues","num_pulls","num_closed_pulls","num_milestones","num_closed_milestones","num_projects","num_closed_projects","is_private","is_empty","is_archived","is_mirror","status","is_fork","fork_id","is_template","template_id","size","is_fsck_enabled","close_issues_via_commit_in_any_branch","topics","trust_model","avatar","created_unix","updated_unix") VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37) RETURNING "id" [1 thepra thepradevhr thepradevhr geeg 0 0 0 0 0 0 0 0 0 0 0 0 true false false false 0 false 0 false 0 0 true false null default<PHONE_NUMBER><PHONE_NUMBER>] - 1.131673ms
2021/02/15 18:23:43 ...m.io/xorm/core/db.go:286:afterProcess() [I] [SQL] ROLLBACK [] - 122.501µs
2021/02/15 18:23:43 routers/repo/repo.go:183:handleCreateError() [E] CreatePost: pq: duplicate key value violates unique constraint "repository_pkey"

After fixing thanks to @zeripath in https://github.com/go-gitea/gitea/issues/14539#issuecomment-779396983 I found that another table has a similar issue through the logs:

2021/02/15 20:41:56 ...m.io/xorm/core/db.go:286:afterProcess() [I] [SQL] INSERT INTO "repo_unit" ("repo_id","type","config","created_unix") VALUES ($1, $2, $3, $4),($5, $6, $7, $8),($9, $10, $11, $12),($13, $14, $15, $16),($17, $18, $19, $20),($21, $22, $23, $24) [29 UnitTypeCode<PHONE_NUMBER> 29 UnitTypeIssues {"EnableTimetracker":true,"AllowOnlyContributorsToTrackTime":true,"EnableDependencies":true}<PHONE_NUMBER> 29 UnitTypePullRequests {"IgnoreWhitespaceConflicts":false,"AllowMerge":true,"AllowRebase":true,"AllowRebaseMerge":true,"AllowSquash":true}<PHONE_NUMBER> 29 UnitTypeReleases<PHONE_NUMBER> 29 UnitTypeWiki<PHONE_NUMBER> 29 UnitTypeProjects<PHONE_NUMBER>] - 516.059µs
2021/02/15 20:41:56 ...m.io/xorm/core/db.go:286:afterProcess() [I] [SQL] ROLLBACK [] - 154.286µs
2021/02/15 20:41:56 routers/repo/repo.go:183:handleCreateError() [E] CreatePost: pq: duplicate key value violates unique constraint "repo_unit_pkey"

Description
After updating to 1.13.2 through stopping gitea, replacing the gitea binary and starting it again I notices that creating a new repository is impossible because the primary keys indices are not aligned, giving the wrong generated ID for the records, it's not only the repository table but also the next called one repo_unit.
At this point what can I do to sync all the indices of the tables?
Is there any command to apply this fix?

Your problem is that your postgres db seems to have the incorrect values in its sequences.
You'll need to run something like:
-- Get Max ID from table
SELECT MAX(id) FROM table;

-- Get Next ID from table
SELECT nextval('table_id_seq');

-- Set Next ID Value to MAX ID
SELECT setval('table_id_seq', (SELECT MAX(id) FROM table));

for each table in postgres.
There is no command to do this.

Have you run gitea dump at some point and then reimported the from that dump?
My suspicion is that the dump is not setting the sequences up correctly.

Have you run gitea dump at some point and then reimported the from that dump?
No, I didn't reimport any dump yet, just made them for backup purposes.

So ok, i'll go for each table with such indexes(Webmin helps here is seeing them) and update the MAX ID.
@zeripath I guess when it comes to binary updates there's nothing really checking/figuring this things out before/after any eventual migration applied(if there's any)?

I'm really not sure what's caused the sequences to disappear like this - maybe they were in the wrong schema?
gitea doctor recreate-table might help?

I don't know if it would help, I did already run all the sets as suggested and I've no more issues, so I'm closing the issue.

Thanks!
It would probably be beneficial to add this to a doctor command given you are the second person with this problem.


add %k and %+k to stack formatter

Pull from golantern/stack, revert name changes back to go-stack/stack.
Tested here and reviewed implementation. Looks like a good approach as the call to .Name() does retrieve the package, filename and other information so it seems to make sense to overload the %n handler to also handle %k.
Goal is to have this available for go-kit/log to retrieve a package name.

Coverage increased (+0.5%) to 88.832% when pulling 3a3e99b4ba4737f629b56b80fe651ae65ea54d58 on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+0.9%) to 89.216% when pulling fa228a6796b776721902144e8b8d86f74687445a on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+0.9%) to 89.216% when pulling fa228a6796b776721902144e8b8d86f74687445a on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+0.9%) to 89.216% when pulling fa228a6796b776721902144e8b8d86f74687445a on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

@ChrisHines changes pushed, waiting for next round of feedback :-)

@ChrisHines tried to rewrite it without any concatenation. Not sure I'm super happy with the outcome but without the string concatenation I'm guessing its more efficient. Net lines of code went up though. Thoughts? I'm still new to go.

Coverage increased (+1.1%) to 89.474% when pulling f3b7c2490066f48b69b6767f8ba178833cabbaa9 on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+1.1%) to 89.474% when pulling f3b7c2490066f48b69b6767f8ba178833cabbaa9 on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+1.1%) to 89.474% when pulling f3b7c2490066f48b69b6767f8ba178833cabbaa9 on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+1.1%) to 89.474% when pulling f3b7c2490066f48b69b6767f8ba178833cabbaa9 on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.

Coverage increased (+0.9%) to 89.216% when pulling<PHONE_NUMBER>1b1917aed9574c25f032c23100004d on chmorgan:develop into 3bd81bdde4686ea4aee961921944d1d012253997 on go-stack:develop.


Feature request: sort by relevance

How hard would something like that be to implement?

I don't plan to add any new functionnality to CommaFeed.
Also, how do you define relevance? :thinking:

Relevance - the thing that Youtube uses to show you videos that you might like to see more than other videos. 😄
I understand that it would require a lot of data mining and some serious algorithms to make that work.
However I didn't know that Commafeed is now feature complete. 👍


Allow _ as a wildcard character in LIKE queries

The SQL standard uses _ as a single-character wildcard. This PR adds in support for _ as a single character by replacing it with . during Regexp construction.

You are awesome. I will look into why the test is failing so we can implement the changes.

I had a quick look at the travis failures. It seemed to be node build issues and dependency issues with node 0.4 to 0.8 rather than test failures. They may have been temporary failures but I couldn't see a way to trigger a rebuild without doing an extra push with an empty commit (which seemed... unappealing :smile: )

I have added a test file of the changes and it looks good. Please accept the pull request at https://github.com/generalassembly/alasql/pull/1
Would be awesome if you could make a pull request to develop...

Ah, you're using a develop branch! I'll close this PR and open a new one for develop.
Thanks for the test file too, all looking good :) Pulled in https://github.com/generalassembly/alasql/pull/1 now so shouldn't take long.

PR is now on the correct branch: https://github.com/agershun/alasql/pull/381


Cloak clips through character when character back to wind

With #34, this is noticeable. This may be an xml issue or something we can fix by just importing the full skeleton of the actor for collisions.
Likely related to #36

testing on an armor mod that have collision body and clipping very small or absent. actually, windStrength=2.0 + SkyrimStormSnow weather might be an extremal condition to test this, and with increasing windStrenght clipping only goes worse

Closing this since we'll likely rely on xml authors to fix this constraint.


test framework todos

[ ] checking for events
[ ] sending of commands
[ ] writing of registers
[ ] dealing with noisy data (approximate matching, use of error register)
[ ] support reading fields of registers
[ ] inherit superclass tests

@pelikhan, @mmoskal - I want to collect ideas for what else may be needed in the test infrastructure.  Of course, this list can grow without bound, so let's also chat to prioritize.

not sure about inheritance of tests. we basically have 1 subclass: sensor. The rest is not shared.

That looks like a great start. You should do a round of writing tests and it will reveal the pieces your are missing.

We should probably move this kind of "issue" into the discussion section.

oh, i forgot about discussions. thanks.


Initialization Error

Hi, I have tried to use your google auth example from examples but I am getting:
`Error: Problem with the flags given to your Elm program on initialization.
Json.Decode.oneOf failed in the following 2 ways:
(1) Problem with the given value:
undefined

Expecting null

(2) Problem with the given value:
undefined

Expecting a LIST`

Elm version: 0.19.1

Seems like a problem unrelated to the library but with some missing flags when initializing the elm application. Looking at the examples, it seems that the actual index.html and the initialization code is not committed on the master branch :s .. but it is on the gh-pages:
https://github.com/truqu/elm-oauth2/blob/gh-pages/auth0/pkce/index.html
As you can see, the examples expect either undefined or a list of bytes to be given as an argument:
    const app = Elm.Main.init({
        node: document.getElementById("app"),
        flags: rememberedBytes()
    });

I am not sure why the index.html wasn't included, probably an oversight. Sorry!

Now versioned on master.
I've also added some instructions about how to build and run the examples locally :+1:


Release prep and bug fixes

Farm Designer:

Add ability to deselect a plant by clicking out of it.

Coverage increased (+0.1%) to 54.225% when pulling 6bdc878ca447529573f8f299947c8d1160b01f1f on gabrielburnworth:master into cd2efd3a2433e98e2f71123a47fdb87c7f5ae359 on FarmBot:master.


Update to<EMAIL_ADDRESS>

I noticed in the grunt 1.0.0-rc release notes that you all are using lodash v3.10.1.
I scanned grunt and didn't see any immediate red flags that would prevent ^4.0.0 so I bumped it.
Let me know if there are issues.

Thanks!

Thank you too. If you run into anything just ping me in.

I'm getting the following when integrating with Grunt:
Warning: grunt.util._(...).chain(...).pluck is not a function Use --force to continue.
TypeError: grunt.util._(...).chain(...).pluck is not a function
  at Object.<anonymous> (/Users/Kyle/Documents/www/grunt/lib/grunt/task.js:260:49)
  at Object.<anonymous> (/Users/Kyle/Documents/www/grunt/node_modules/grunt-contrib-jshint/tasks/jshint.js:44:21)
  at Object.<anonymous> (/Users/Kyle/Documents/www/grunt/lib/grunt/task.js:264:15)
  at Object.thisTask.fn (/Users/Kyle/Documents/www/grunt/lib/grunt/task.js:82:16)
  at Object.<anonymous> (/Users/Kyle/Documents/www/grunt/lib/util/task.js:301:30)
  at Task.runTaskFn (/Users/Kyle/Documents/www/grunt/lib/util/task.js:251:24)
  at Task.<anonymous> (/Users/Kyle/Documents/www/grunt/lib/util/task.js:300:12)
  at /Users/Kyle/Documents/www/grunt/lib/util/task.js:227:11
  at nextTickCallbackWith0Args (node.js:419:9)
  at process._tickCallback (node.js:348:13)
  at Function.Module.runMain (module.js:444:11)
  at startup (node.js:136:18)
  at node.js:966:3

About to look into why but maybe you know off the top of your head?

Ah ha, removed in favor of .map(): https://github.com/lodash/lodash/issues/1806

Ah ha, removed in favor of .map():

Ah yep, I scanned for _.xyz use and missed chaining.


Way to tell TS that a JavaScript variable really is used somewhere

From https://github.com/Microsoft/vscode/issues/51405
Feature request
For JavaScript, add a way to mark that a variable that appears unused is used somewhere. For example it may be used within a template that TS cannot analyze
Not sure if @ts-ignore should suppress unused too. It currently has no effect:

@andy-ms What about AngularJS? There are cases when you have a variable in the controller, which is used indirectly in the scope:
<!-- my-thing.component.html -->
<div>{{ $ctrl.property }}</div>

// my-thing.component.ts
class MyThing {
  public property = 42; // marked as unused, but is really used in the view
}

@parzh Are you sure there isn't some other tool marking that as unused? TypeScript should never mark anything public as unused.

@andy-ms Oh, I actually didn't test it. Sorry for the confusion 😔
I just meant that there are cases where it might actually be useful to indicate explicitly that a locally scoped entity is used, even if it appears as unused. A lot of those are related to frameworks with view separated from controller (like AngularJS), but unfortunately  I gave a bad example.

As far as I can tell, there's no case where unused detection has a false positive (other than something erroneously marked private) -- let me know if you find a counterexample.


Handling operators default implementation

An operator must have an LLVM-based implementation at the core, so there must be clear guidelines for implementing this. This issue will be a discussion on the topic of possible implementations of default operator handling and operator overloads.
Without a default implementation, there is no access to the LLVM functions resulting in a nonfunctioning language.

One possibility is to group a data type such as the numeric signed and unsigned into a trait which operators have definitions for but without a declaration as that is handled upon that specific argument list within the compiler.
Alternatively, there can be a language feature to access the LLVM function and some compiler operations that access the necessary values like the last register value. A compiler operation is for example #sizeof(Type) that gives the byte size of a specified type.

There is now an idea of how this will be handled. All operators are meant to be defined somewhere by Ferrum source code, this might come with some exceptions like the CALL operator. This is because there is no nice way to define this in source code as far as I can think of right now. Might come back to this at a later date to rethink some operators.


WFLY-20023 Remove all non-breaking uses of ModuleIdentifier in Concurrency

Issue: https://issues.redhat.com/browse/WFLY-20023

Simple view of TC test status (mostly for my own benefit this today): https://ci.wildfly.org/project.html?projectId=WF_PullRequest&branch_WF_PullRequest=pull%2F18546

Thanks @RanabirChakraborty and @ropalka


Connection heartbeat arg description doesn't match implementation

The heartbeat arg description https://github.com/agoragames/haigha/commit/73cd9b1736fb72f3a140b6317b526a2111e38a7f#diff-a440ae8b97ad1a5653b91743003e498aR334 is

heartbeat Default None (disabled). If 0, broker assigned. If >0, negotiated with broker.

The implementation appears to be that if user passed heartbeat=None, then the heartbeat value is broker-assigned; if user passed heartbeat=0, then heartbeats would be disabled. The quoted description is the reverse of what actually happens in the code.
        if self.connection._heartbeat is None:
            self.connection._heartbeat = method_frame.args.read_short()

CC @josegonzalez

I don't know. It seems like the documentation is right, but its hard to figure it out based on the naming of the heartbeat variable across the codebase.

The negotiation logic is:
        if self.connection._heartbeat is None:
            self.connection._heartbeat = method_frame.args.read_short()

which accepts broker's value if user's value is None. However, the description says "Default None (disabled).", which is absolutely not the same thing.

The logic here actually doesn't send the heartbeat either way though?
        if self.connection._heartbeat:
            if time.time() >= (self._last_heartbeat_send + 0.9 *
                               self.connection._heartbeat):
                self.send_frame(HeartbeatFrame(self.channel_id))
                self._last_heartbeat_send = time.time()

Correct, but the broker cannot pass the value None in Connection.Tune; that value None could only have come from the user. If the user passes None, the only way that self.connection._heartbeat could end up as 0 after negotiation is if the broker passed 0 in Connection.Tune

True, but isn't _heartbeat set by the consumer?

It's set from user's arg here: https://github.com/agoragames/haigha/blob/a3ce85536f3de902ef1056304237e873cde6a8fe/haigha/connection.py#L67 and then overridden here https://github.com/agoragames/haigha/blob/a3ce85536f3de902ef1056304237e873cde6a8fe/haigha/connection.py#L608

@vitaly-krugl note: since we deployed this change in production, we no longer get heartbeat timeouts in related services. This definitely merits further research though.

@josegonzalez, this makes sense, because your change heartbeat=0 disables heartbeats on both client and broker sides of the connection.

So are the docs wrong?

@josegonzalez, yes the docs are wrong, and that's the gist of this issue.

@vitaly-krugl so one reason we made this change was an upgrade to RabbitMQ moved the timeout from 5 minutes to 1 minute, impacting some of our longer running jobs (map and ticket processing). What you're saying makes it sound like there is an issue in our networking in regards to sending the heartbeats maybe, especially if the default of None should make it respect server heartbeats...

Hmmm, seems like us not sending heartbeats was unrelated. So then yes, the docs here are almost certainly wrong, heh.


Unicode symbols cause problems with PyJulia

This is related to #116
I think we should provide options to access all RHEOS functionalities without having to use Unicode symbols.
This would involve:

providing alternative names for passing model parameter values, eg eta or c_beta, and let RHEOS convert the symbols where appropriate.
allowing several keyword parameters for certain functions, such as importcsv and RheoTimeData constructor.
providing functions to access the stress and strain fields of RheoTimeData, and in other relevant places.

One option for 1 is to have a const named tuple with a symbol conversion table, and then convert symbols in check_and_reorder_parameters
julia> d=(eta=:η, c_alpha=:c_α)
(eta = :η, c_alpha = :c_α)

julia> nt = (k=1.0, eta=2.0, c_alpha=4.5)
(k = 1.0, eta = 2.0, c_alpha = 4.5)

julia> NamedTuple{Tuple([ s in keys(d) ? d[s] : s for s in keys(nt) ])}(values(nt))
(k = 1.0, η = 2.0, c_α = 4.5)

For 2, this could maybe be achieved like this:
function RheoTimeData(;strain = RheoFloat[],  ϵ::Vector{T1} = strain, stress = RheoFloat[], σ::Vector{T2} = stress, ...

Here is a bit of code that now works fine without using unicode symbols.
using RHEOS
using Plots

data = importcsv("C1_S_0A5_90_rheos.csv", time = 1, strain = 2, stress = 3)
plot(gettime(data), getstrain(data), getstress(data))

model=modelfit(data, FractD_Maxwell, stress_imposed, p0=(beta=0.05, c_beta=0.05, eta=10.0))

print(getparams(model, unicode=false))
# (eta = 65.17997591394088, c_beta = 0.08515319205963537, beta = 0.08989064046489526)

data_stress = extract(data, stress_only)

data_fit = modelpredict(data_stress, model)

plot(gettime(data_fit),[getstrain(data_fit), getstrain(data)])


refactor(net): move node record constants to network-peers crate

Should close #9157.

Can you make it slightly more structured?
bootnodes/mod.rs
bootnodes/ethereum.rs
bootnodes/optimism.rs

Probably have to fix a bunch of imports across different crates as well

fixed


Add browser for object store

Description
This PR...

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

@blueorangutan package

@kishankavala I am merging my branch into your branch (csos). It's bit difficult to test directly on this PR.


Reuse timer in backend.run.

Hi!
This is my first and pretty trivial attempt to contribute to etcd. Was reading code and this particular piece of code made me interested. It runs every 100ms and starts new timer on every iteration. Seems that it is a bit wasteful and can be easily avoided.
I guess producing 30 objects/s is not that much, but it is a low hanging fruit, so I took a bite.
Feel free to reject it.
Benchmarks:
import (
	"testing"
	"time"
)

func BenchmarkTimeAfter(b *testing.B) {
	b.ReportAllocs()
	for n := 0; n < b.N; n++ {
		select {
		case <- time.After(1 * time.Millisecond):
		}
	}
}

func BenchmarkTimerReset(b *testing.B) {
	b.ReportAllocs()
	t := time.NewTimer(1 * time.Millisecond)
	for n := 0; n < b.N; n++ {
		select {
		case <- t.C:
		}
		t.Reset(1 * time.Millisecond)
	}
}

Running reveals that each loop results in 3 allocs:
BenchmarkTimeAfter-4 	    2000	   1112134 ns/op	     192 B/op	       3 allocs/op
BenchmarkTimerReset-4	    2000	   1109774 ns/op	       0 B/op	       0 allocs/op

Can you change the commit message? Something like backend: ...?
Defer to @xiang90 @heyitsanthony

Can you change the commit message?

Done!

@nekto0n Use https://golang.org/pkg/time/#NewTicker instead?

Yeah, I looked at it, but not sure if it fits. I though that semantics of this particular loop is sleep for N seconds (or more), but with ticker I guess we'll sleep for (N - commitTime), i.e. if commit took 99ms ticker will fire after 1ms.

@nekto0n You are right. Timer is more accurate. LGTM.

Fix the stop thing? Then let's get this merged.

Done.


How to only get the the artifacts for the current folder

Greetings, your script is awesome.
I plan on putting in thousands of files in subfolders of subfolders.
How can I get the file list for a particular ID, and NOT go recursive?
Thanks!

I apologize for the inconvenience. Although I'm not sure whether I could correctly understand your question, if you want to retrieve only files in a folder  without using the recursive function, please use the list method of Drive API. At that time, please use '### folder ID ###' in parents as q of search query. If you want sample scripts, you can see them at Stackoverflow.

list method of Drive API
Search for Files


Times corpus filter not working

The filter 'On front page' True/False is not working properly anymore. When selecting the filter, i.e. set it to True, it shows 0 entries.

Disabled the filter for now


Special logger for JsonRpcEndpoint

A RequestResponseLogger capable of buffering and counting repeated requests and
responses was created and plugged into JsonRpcEndpoint to reduce noise
in the logs.
Fixes #121

Repeated 11 times over 0 seconds: {
  request[backgroundTasks]: {"data":null}
  response: {"data":["ProgressIndicator 441017100: running=true; canceled=false"]}
}

Found this in your log, perhaps we should have special variant of message for 0 seconds and 1 second

Good, to merge I think.
I looked into CI logs, and I have one last reflection. We group it with C syntax. WDYT about Python? Maybe it would be more clear:
Repeated 10 times over less than second {
  request[a]: {}
  response: []
}
request[a]: {}
response: []
Repeated 10 times over less than second {
  request[a]: {}
  response: []
}
request[a]: {}
response: []

vs
Repeated 10 times over less than second:
  request[a]: {}
  response: []
request[a]: {}
response: []
Repeated 10 times over less than second:
  request[a]: {}
  response: []
request[a]: {}
response: []

I am not sure myself...

Wow @kszydlo-vl you're contributing here as well? 😯

Actually, out of curiosity, when is this change going to be released?

Hopefully tomorrow in maven central as 0.9.0


feat: Limit CPU in runtime (#3685)

I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
https://github.com/GreptimeTeam/greptimedb/issues/3685
What's changed and what's your intention?

Add parking_lot in root Cargo.toml, use parking_lot.workspace = true in sub project.
Expand the runtime with optional resource limitation level.

Checklist

[ ] I have written the necessary rustdoc comments.
[ ] I have added the necessary unit tests and integration tests.
[ ] This PR requires documentation updates.

@fengys1996 Please also take a look at this PR, aiming to provide a new relaxed runtime to constrain CPU usage of spawned tasks.

It looks like there are some conflicts and some compilation issues.
Please resolve them first.

@fengys1996 I think we can test this feature on the new version of GreptimeDB edge now to see if it can manage the total CPU usage.

Great work! Thanks a lot.


Ensure the loading of the product example during onboarding does not block the UI

What

Ensure the loading of the product example during onboarding does not block the UI

Part of

#561

@teolemon What are we supposed to display while the data is being downloaded and if it's taking too long? How long is "too long"?

@teolemon That would mean that if there's no (or poor) internet connection we display the default value, i.e. en_US version?

Please assign me this issue, if this is open

Sure @atharv028

@atharv028 I'm unassigning you. Feel free to reclaim if you're still working on it

Not relevant anymore as we already use a 5 second time-out, and in the big redesign we don't use the product example.


Include Author's Notes with chapter scraping

The chapter scraper includes the Author's Note section from each chapter in the downloaded source files. This inflates the word counts and also includes TextRef(s) that aren't strictly part of the main story.

The scraper should either detect the Author's note and skip it or split the source files in post processing

Note the scraper also fails to capture earlier Author's Notes that were simply the first comment on the post and not embedded directly into the chapter.

It may be better to use the Wordpress API to grab comments although with the new website, this may not be necessary.


remove dashboards from group-detail serializers

Dashboards are not needed in group detail

Pull request made 8/21/2019
#47

Merged to master 8/21/2019


PickerIOS React.createElement: type should not be null, undefined, boolean, or number.

As stated in the title i receive this when using the PickeriOS. It gives me the warning whenever I initially run the application or when the picker is moved. the Picker code is as follows.
<PickerIOS
 selectedValue={this.state.OccuType}
 onValueChange={(OccuType) => this.setState({OccuType})}>
    {Object.keys(OccupancyType).map((name) => (
         <PickerItemIOS
            key={name}
            value={name}
            label={OccupancyType[name].name}
             />
))}
</PickerIOS>

The code for the objects that it is using is:
var OccupancyType = {
    Residential: {
      name: 'Residential'
    },
    Multi: {
      name: 'Multi-Unit Residential'
    },
    Public: {
      name: 'Public Assembly'
    },
    Strip: {
      name: 'Strip Mall'
    },
    ComInd: {
      name: 'Commercial/Industrial'
    },
    High: {
      name: 'High Rise'
    }
};

Now the Picker works. Heres an image. Heres the warning that it gives me.
I'm am currently on React Native ^0.24.1 and developing on a Mac for an iOS device.

What does the render method of VandenbergFire look like?

@facebook-github-bot close

@facebook-github-bot no-reply

@facebook-github-bot no-reply

Closing this issue as more information is needed to debug this and we haven't heard back from the author. Once there's more information we can reopen the issue.

import {View,PickerIOS,Text,PickerItemIOS} from 'react-native';
causes this issue
fixed by
import {View,PickerIOS,Text} from 'react-native'; var PickerItemIOS = PickerIOS.Item


Allow for naming the rootId with no options

Right now if you want to name your rootId something different, but don't pass
in any options, it doesn't change the name of your rootId (see confusion in
github #49.)
This would allow someone to use
const app = choo()
app.start('myRootNode')

Rather than having to use
const app = choo()
app.start('myRootNode', {name: 'myRootNode'})

ahh dammit - great catch!

released as v2.2.2 :sparkles:


python3Packages.schemdraw: 0.17 -> 0.18, update dependencies

Description of changes
All packages by the same author that depend on each other, so I collated their updates into one PR.
https://github.com/cdelker/ziafont/compare/0.6...0.7
https://github.com/cdelker/ziamath/compare/0.8.1...0.9
https://github.com/cdelker/schemdraw/compare/0.17...0.18
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 279462 run on x86_64-linux 1

12 packages built:

python310Packages.schemdraw
python310Packages.schemdraw.dist
python310Packages.ziafont
python310Packages.ziafont.dist
python310Packages.ziamath
python310Packages.ziamath.dist
python311Packages.schemdraw
python311Packages.schemdraw.dist
python311Packages.ziafont
python311Packages.ziafont.dist
python311Packages.ziamath
python311Packages.ziamath.dist

@ofborg build python311Packages.schemdraw

Packages broke on master, added fixes and rebased.

Result of nixpkgs-review pr 279462 run on x86_64-linux 1

6 packages built:

python311Packages.schemdraw
python311Packages.schemdraw.dist
python311Packages.ziafont
python311Packages.ziafont.dist
python311Packages.ziamath
python311Packages.ziamath.dist

Result of nixpkgs-review pr 279462 run on x86_64-linux 1

6 packages built:

python311Packages.schemdraw
python311Packages.schemdraw.dist
python311Packages.ziafont
python311Packages.ziafont.dist
python311Packages.ziamath
python311Packages.ziamath.dist

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/3334


Cmorize gpcp

Description
This pull requests adds scripts to cmorize latest monthly mean GPCP-SG data from NOAA.
Data can be accessed here: wget https://downloads.psl.noaa.gov/Datasets/gpcp/precip.mon.mean.nc
There is also GPCP-SG data in Tier 1, but restricted to the years 1979 - 2017. The dataset above is constantly extended and currently covers 1979 - (Jan) 2023.
I applied the cmorized data in the diagnostic validation.py and compared it to precipitation data from ERA5.

Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[ ] 🛠 Code is written according to the code quality guidelines
[x] 🛠 Tests run successfully
[ ] 🛠 All checks below this pull request were successful

New or updated data reformatting script

[x] 🛠 Documentation is available
[x] 🛠 The dataset has been added to the CMOR check recipe
[x] 🧪 Numbers and units of the data look physically meaningful

Okay, automatic download is working. Thank you very much!

Hello, I saw that this merge request is still open. Some code check failed. However, I do not know how to fix these issues. Codacy complains that there are unused arguments, but I cannot remove these, since they are called by the main cmorizer that way even though the arguments are not necessary for this dataset download and cmorization. Could somebody help me fixing that? Thank you very much!

You can ignore the Codacy issues for this (these cannot be solved for the CMORizers). I am not quite sure why the documentation build failed, but it could be related to a change to the theme in ESMValCore lately. We will update that also for ESMValTool soon, so hopefully this will run fine once we did that!

Thank you very much for taking the time to review this. I accepted your suggestions, thank you.

Thank you very much for taking the time to review this. I accepted your suggestions, thank you.

You're very welcome, @FranziskaWinterstein! All good on my side. I'll let @schlunma merge this PR if all addressed on his side as well 👍


Increase in compilation times with newer releases

There is a noticeable increase in the compilation time of models with newer pybuda releases.
For example (BERT-Large w/ 384 input size):

Release Version
Release Date
Compilation Time (s)
Performance (sen/s)

v0.9.80
Jan 5, 2024
117.7
65

v0.12.3
May 10, 2024
261.8
114

v0.18.2
Jul 18, 2024
358.4
118

That is a 3x increase in compilation time.
Some models have been reported to take even longer going from 5min -> 60min.

We will monitor this for now for severity.


[rtmidi] init port #582

@atkawa7,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

Thanks for the PR!
I made a few simplifications to the CMakeLists.txt while merging:

We automatically define BUILD_SHARED_LIBS since it's a core CMake variable
BUILD_SHARED_LIBS is automatically used to select STATIC or SHARED for calls to add_library()
Most paths are automatically prefixed by the current path to the CMakeLists.txt, so adding ${PROJECT_SOURCE_DIR} isn't needed

@ras0219-msft Thanks. Good to know. I am learning


Quick actions menu

Feature Description

Some actions are cumbersome in the editor as it's difficult to find them in the Design panel. As such, we should implement a quick actions menu that shows up when an element is selected, with the appropriate actions easily accessible by the user.
Key actions to include:

Change color (text, shapes)
Change font (text only)
B/I/U (text only)
Crop (media only)
Add link

Some of these elements may not be necessary if they are present in the Right-click/Context menu #97
Alternatives Considered

Additional Context

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria

Implementation Brief

Hi @o-fernandez @choumx @bmattb  ,
I'm sharing the Figma file with the right Quick Actions Menu design and flows. We will have a hand-off session this week to go over this, but in the mean time feel free to add your comments/questions:
https://www.figma.com/file/NHHMwIfxMnz39NxqkrFb7R/Stories---Post-Stable?node-id=4462%3A0

Hi @o-fernandez @choumx @bmattb  ,
I'm sharing the Figma file with the right Quick Actions Menu design and flows. We will have a hand-off session this week to go over this, but in the mean time feel free to add your comments/questions:
https://www.figma.com/file/NHHMwIfxMnz39NxqkrFb7R/Stories---Post-Stable?node-id=4462%3A0

Generally these look good (both right click and context), but we should iterate on sequencing and scope based on eng feedback.

Hey @bmattb - this ticket is duplicative of the epic. Do you want to keep it around or close it?

Closing as duplicate of #6147


UC Browser & Nokia Xpress browser support, browser.proxy? browser helper method

Added detection for:

UC Browser
Nokia Xpress Browser (previously known as Nokia S40 Ovi Browser.  Nokia/Microsoft have not changed user agent to reflect name change)

Also added a browser.proxy? method that returns true for following browsers:

UC Browser
Nokia Xpress Browser
Opera Mini

Specs added to test new functionality.
Why? Because popular front end techniques and frameworks work poorly through use of  proxy browsers (e.g. Bootstrap), its nice to know to be able to offer alternative UI for these users.

Any chance of this getting merged?

Thanks for your PR. Too much time has passed (my bad) and it was impossible to merge it. I copied the tidbits, which are available on these commits:

Proxy browsers: 0c9415002c1301678037c0786a8e8cb2b3bdcd79
Nokia S40: dc526b76d9bef45a419a663927b17dfe3f5938d2
UCBrowser: dfa59859df35c5194378353b7b1d530e81ff201a


Add custom control option to select

Add the ability to specify that the given select is a custom control,
adding the necessary classes.

Pull Request Test Coverage Report for Build 107

5 of 5 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 106:
0.0%

Covered Lines:
810

Relevant Lines:
810

💛  - Coveralls

I think it looks good. Please fix the rubocop warning and add an example in the demo gauntlet file: https://github.com/comfy/comfy-bootstrap-form/blob/master/demo/app/views/bootstrap/form.html.erb#L135 Thanks! 👍

@GBH I've fixed the related Rubocop warnings now. 🙂

Added select-example in bd62bf6f5c8fff456709a0881d442274251a3d63.

Good stuff. Thanks!


compiled release

Hi,
i'm new to serialization since today. I saw a great link in StackOverFlow comparing a lot of serialization libraries. Protostuff did really well on those benchmarks. And often i'm not a huge van of the default java libraries.
I only wonder some things about protostuff:
1). Why isn't a compiled release included in the download? (and what is the easiest way to compile?)
2). Do I only need the core? And if not, what folders are important, there are 18 of them...
3). Do I need to have maven?
I hope someone can help. A link to some good tutorial is also welcome but I couldn't find any descent tutorial.

Hi @clankill3r
It is a best practice to use build system, for java we have many of them - the most popular are - maven and gradle.
Here is an example how to use protostuff with maven: https://github.com/protostuff/protostuff-examples/tree/master/hello-world
If you want store jar files locally, then you can find all released artifacts in maven-central: http://mvnrepository.com/artifact/io.protostuff.
For example, protostuff-core is here: http://central.maven.org/maven2/io/protostuff/protostuff-core/1.3.6/protostuff-core-1.3.6.jar
Compilation is trivial - you should install maven and run following on project root:
$ mvn clean install

Thanks for the reply.
What makes that example a maven example? I can't find anything in HelloService.java related to maven.

What makes that example a maven example?

pom.xml

@clankill3r What would you like to see in the tutorial?
I'm going to write tutorial on the next week - basically I will put all existing documentation to one place. I can add some things that looks obvious to me, but are not obvious to people new to java.


[Bug] Multi-word keywords are truncated

Issue
When upload a CSV file of keywords composed of multiples works, the content is stored incorrectly:
Content of the uploadedCSV file
project management tools, product management software

Spaces are removed

Expected
Multi-word keywords should be stored and processed as-is

The implementation in https://github.com/rtp-atw/nimble-interview/pull/25 is rather hard to understand 😅
Instead of trying to support multiple formats (CSV in a single line and CSV with line breaks), it would be acceptable to support only one format and provide a template (see below) 💡


question about function Goban::is_eye()

in function:  bool Goban::is_eye(Point pt, Color player) const
the original code:
        if (board.countEqual(corners, -player) >= (corners.size >> 1) && board.getMinLibertiesOfSurroundingGroups(pt) <= 1) {
i think it should be
        if (board.countEqual(corners, -player) >= (corners.size >> 1)   || board.getMinLibertiesOfSurroundingGroups(pt) <= 1) {
any condition may cause false eye, so it should be ||, not &&?

Doing so causes a lot of tests to fail, so my immediate hunch is "nope"

thank you


"dotnet tool install/update" not working with Azure Artifacts

Note
Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo
For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
If you have an issue or request for the Azure Pipelines service, use developer community instead:
https://developercommunity.visualstudio.com/spaces/21/index.html )
Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Question or Feature request
Enter Task Name: DotNetCLI@2
list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
Environment

Server - Azure Pipelines
Agent - Private: Windows Server

Issue Description
dotnet tool install (invoked as custom command via DotNetCLI@2) cannot install tool when NuGet points to Azure Artifacts.
We've got an Azure Artifacts feed for our packages that is configured to use the NuGet gallery as upstream source. We've created a NuGet.config file pointing to the Azure Artifacts feed only. Restoring packages using the restore command works, however trying to restore a tool via a custom command does not. Trying to retrieve information about the package results in a 401 (Unauthorized).
Looking at the implementation of DotNetCLI@2 it looks like there are special cases for dotnet pack, dotnet push and dotnet restore among other things handling implicit authentication. However there is not special treatment for dotnet tool install or dotnet tool update.
Feature Request: Handle dotnet tool install and dotnet tool update or make the authentication globally available to all dotnet commands.
Question: Is there a workaround for this scenario (other than not removing nuget.org via NuGet.config, which would be against the point of using Azure Artifacts with upsteram nuget.org)?

Hey @michael-x
Are you able to install the tool via "dotnet tool install" command with same nuget.config on a local setup using same sdk version?
If not, then it is most likely the dotnet sdk issue, as been discussed in the issue thread on dotnet cli repo: https://github.com/dotnet/cli/issues/9586 . According to the thread, the fix for this is in 3.0.100 preview and later.
Hope this helps

@hiyadav, if I understand things correctly, the ticket that you referenced is about a situation where dotnet tool install doesn't work, because there are two feeds in NuGet.config and a feed that requires authentication (but is not authenticated) blocks installation, even if the tool comes from the other feed that does not require authentication.
This is more about the fact that DotNetCLI@2 does some magic for the "pack", "push" and "restore" commands to provide authentication for Azure Artifact feeds (and maybe others, don't know) that require authentication, but it doesn't do that for "tool install" or "tool update".

That's the bug I'm experiencing @michael-x
2>C:\Program Files\dotnet\sdk\3.0.100-preview6-012264\NuGet.targets(123,5): error : Unable to load the service index for source https://myorg.pkgs.visualstudio.com/_packaging/dotnet-platform/nuget/v3/index.json. [C:\Users\Me\AppData\Local\Temp\lvpavr0a.4ul\restore.csproj]
2>C:\Program Files\dotnet\sdk\3.0.100-preview6-012264\NuGet.targets(123,5): error : Response status code does not indicate success: 401 (Unauthorized). [C:\Users\Me\AppData\Local\Temp\lvpavr0a.4ul\restore.csproj]
2>The tool package could not be restored.
2>Tool 'dotnet-ef' failed to install. This failure may have been caused by:
2>
2>* You are attempting to install a preview release and did not use the --version option to specify the version.
2>* A package by this name was found, but it was not a .NET Core tool.
2>* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
2>* You mistyped the name of the tool.
2>C:\dev\demo\source\Demo.Data\Demo.Data.csproj(31,9): error MSB3073: The command "dotnet tool install --local dotnet-ef --version 3.0.0-preview6.19304.10" exited with code 1.
2>Done building project "Demo.Data.csproj" -- FAILED.

Me to, so count me in on a fix:-)

`PM> dotnet tool install --global dotnet-ef
C:\Program Files\dotnet\sdk\2.1.801\NuGet.targets(123,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/notalib/_packaging/Dependencies/nuget/v3/index.json. [C:\Users\pmd\AppData\Local\Temp\rgjvalkm.rsi\restore.csproj]
C:\Program Files\dotnet\sdk\2.1.801\NuGet.targets(123,5): error :   Response status code does not indicate success: 401 (Unauthorized). [C:\Users\pmd\AppData\Local\Temp\rgjvalkm.rsi\restore.csproj]
dotnet : The tool package could not be restored.
At line:1 char:1

dotnet tool install --global dotnet-ef

  + CategoryInfo          : NotSpecified: (The tool package could not be restored.:String) [], RemoteException
  + FullyQualifiedErrorId : NativeCommandError

Tool 'dotnet-ef' failed to install. This failure may have been caused by:

You are attempting to install a preview release and did not use the --version option to specify the version.
A package by this name w
as found, but it was not a .NET Core tool.
The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
You mistyped the name of the tool.
`

Hey,
We have added it to our backlog and will pick it up in coming sprints. I'll post about further update soon.

Is it possible to suggest a workaround?
Would be appreciated very much, thanks

Fra: himanshu yadav<EMAIL_ADDRESS>Sendt: 26. september 2019 10:14:05
Til: microsoft/azure-pipelines-tasks
Cc: Peter Stjernholm Meldgaard; Comment
Emne: Re: [microsoft/azure-pipelines-tasks] "dotnet tool install/update" not working with Azure Artifacts (#10057)
Hey,
We have added it to our backlog and will pick it up in coming sprints. I'll post about further update soon.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/azure-pipelines-tasks/issues/10057?email_source=notifications&email_token=ABMIZXUVDA4NKPDMFV7RTN3QLRVM3A5CNFSM4HEKQ342YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7UXG4A#issuecomment-535393136, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABMIZXUKP4MIREYKTHAVHITQLRVM3ANCNFSM4HEKQ34Q.

Ok, just an update, because this problems expands as you search for a workaround.
It seems like the dotnet commands ignores the dropdown selections for "Package source" and "Default Project" in VS 2019 package manager console.
So it does not take these into account when executing a command like: dotnet tool install --global dotnet-ef
Trying to specify a source (-add-source) on the command line does neither help.
What I did then was to go to options->Nuget Pack..->Pack. sources and remove the entry pointing to our Azure repository. This helped and dotnet could now install the dotnet-ef tools.
Now trying to install a package as:  dotnet add package Microsoft.EntityFrameworkCore.Design
result in: dotnet : Could not find any project in C:\Projects\Watermark\WatermarkSystem\Application\WatermarkSystem\.
So it does not take the "Default project selecting into account", ok so i CD to my project folder and repeat the command:
PM> dotnet add package Microsoft.EntityFrameworkCore.Design
  Writing C:\Users\pmd\AppData\Local\Temp\tmp2FC4.tmp
info : Adding PackageReference for package 'Microsoft.EntityFrameworkCore.Design' into project 'C:\Users\pmd\source\repos\TestEFCore\TestEFCore\TestEFCore.csproj'.
info : Restoring packages for C:\Users\pmd\source\repos\TestEFCore\TestEFCore\TestEFCore.csproj...
error: Unable to find package Microsoft.EntityFrameworkCore.Design. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages
error: Package 'Microsoft.EntityFrameworkCore.Design' is incompatible with 'all' frameworks in project 'C:\Users\pmd\source\repos\TestEFCore\TestEFCore\TestEFCore.csproj'.

Incompatible with what? This is an empty project with core 3.0 as target platform? How could it be incomp.?

To get unblocked you would need to prefill authentication in the nuget file for the feed that is to be used.
You can edit the nuget file using a scripting task like PowerShell or you can use NuGetAuthenticate tool for updating the nuget file with required authentication

@Hiyadav What?? Wrong issue or ?

@elbatk is this something you should consider?

I too have the same issue. I have published a custom dotnet tool as nuget package to our AzureDevops artifacts. But when try to install the tool using dotnet tool install command I get unauthorized error.

Ok, for all that have this issue, I have a workaround so at least you can get your pipeline running.
I go one directory up and then install, so no nuget.config issues:
- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      cd ..
      dotnet tool install -g dotnet-ef
      export PATH="$PATH:$HOME/.dotnet/tools/"
      cd $(build.sourcesdirectory)
      dotnet ef migrations script --idempotent --no-build --configuration $(buildConfiguration) -p <MyProject> -o $(build.artifactstagingdirectory)/migrations.sql

I am having this issue now and its frustrating. Why cant the dotnet tool restore work the exact same way as dotnet restore? I dont want to bake PATs into my Nuget.Conf, I don't want to include a hack task.

@tejasd1990 would it be possible to use NugetAuthenticate task along with CLI task to solve this problem?

@bishal-pdMSFT
I tried this:
steps:

- task: NuGetAuthenticate@0

- task: DotNetCoreCLI@2
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'restore --interactive'

Got this:

C:\Program Files\dotnet\sdk\3.1.202\NuGet.targets(124,5): error : Problem starting the plugin 'C:\Users\VssAdministrator.nuget\plugins\netcore\CredentialProvider.Microsoft\CredentialProvider.Microsoft.dll'. Plugin 'CredentialProvider.Microsoft' failed within 5.657 seconds with exit code -1. [C:\Users\VssAdministrator\AppData\Local\Temp\nx1ok0f3.nhz\restore.csproj]

This worked for me:
steps:

- task: NuGetAuthenticate@0

- task: UseDotNet@2
  displayName: Install .NET Core
  inputs:
    useGlobalJson: true

- task: CmdLine@2
  displayName: 'Execute Cake Script'
  inputs:
    script: |
      dotnet tool restore --interactive

Same problem, this was my workaround:
  - script: |
        dotnet tool restore  --add-source https://api.nuget.org/v3/index.json --ignore-failed-sources
    displayName: Restore .NET Tools

@henriksen --ignore-failed-sources clever, thanks!
I will say one downside is the tool isnt fetched through the Artifacts source so it wont be copied to artifacts forever.

I will say one downside is the tool isnt fetched through the Artifacts source so it wont be copied to artifacts forever.

That's true, but for us that's not a huge problem since the tool we need is Entity Framework, and if that suddenly goes away, we're gonna have bigger problems 😁 YMMV

Not sure if it was the right way to solve the problem, but I submitted a PR to add tool install as a command on the DotNetCoreCLI task with functionality similar to how the restore command words (I.E. selecting a feed to use). Worked like a charm when I tested so let's see what happens.

This is still a problem. So not stale!

@andrewdmoreno : has your PR been accepted yet? Can you please link it here so we can check it out?

@mvonballmo - No, it has not unfortunately. It's linked above my previous comment.

@andrewdmoreno : Thanks for the quick reply. I see that integration has completely foundered on that PR.
For those that come after: I was able to strip down a workaround from the PR comments a bit more. The following steps worked for me:
  - task: NuGetCommand@2
    displayName: 'NuGet Add Credentials For Internal Feed'
    inputs:
      command: custom
      arguments: 'sources add -Name "<INTERNALFEEDNAME>" -Source "<INTERNALFEEDURL>/nuget/v3/index.json" -Username "this_value_could_anything" -Password "$(System.AccessToken)"'
  - task: Bash@3
    displayName: Install tools
    inputs:
      targetType: inline
      script: dotnet tool install <TOOLNAME> --global

Where:

<INTERNALFEEDURL> is obtained from your Azure project
<INTERNALFEEDNAME> doesn't matter, as long as it doesn't conflict with any defaults
<TOOLNAME> is the name of the tool to execute

This is utterly unintuitive, but it works and it's not too much hacking. At least I can copy/paste these build steps between projects that rely on my dotnet tool without having to worry about copying secret information or uploading secure files. The idea to use $(System.AccessToken) was great.
I think it's indisputable that it would be much nicer if "install tool" was an option for the "dotnet" command. It's not like it's an external tool. This is literally how Microsoft has asked us to work.
It would be nice if I hadn't had to spend half an afternoon trying to figure out how to get a dotnet tool installed from a feed in the same project on Azure. I'm glad I got it working, but everyone who comes after will also waste time trying to figure this out---or will give up and use a gross hack instead.

Another workaround that is a bit iffy but works, you can install tool with a custom nuget.config. what I did was check in a default.nuget.config that only points to standard feed. Then you don't need these steps.

For all those that come after me:
I will be posting this response on a few issues that I frequented when troubleshooting a similar issue. This is not strictly related to this particular issue but someone may find it useful in the end.
In my case, we were having issues with intermittent packages not being restored from a nuget feed with a networking error, for example:
  Failed to download package 'System.Data.OleDb.4.7.0' from 'https://pkgs.dev.azure.com/wello-ca/af808fc2-ae7d-42da-887a-ea7a78a65e52/_packaging/7648731a-3d0b-4888-816b-d229e1e80fd4/nuget/v3/flat2/system.data.oledb/4.7.0/system.data.oledb.4.7.0.nupkg'.
  The SSL connection could not be established, see inner exception.
    Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
    An existing connection was forcibly closed by the remote host.

Multiple packages would fail, and the packages that failed would be different every time. We would also periodically see a dotnet tool restore command simply hang and never complete.
We eventually figured out that these issues only seemed to be occurring when multiple builds were happening in parallel on Azure DevOps. This led me to believe that perhaps there was some kind of rate-limiting happening against the Azure Artifacts server, because the solution in question had ~70 projects to be restored, thus making a lot of network calls.
I enabled caching of nuget packages to alleviate this, which seemed to solve the problem. Below is a template YAML that I have added for all build tasks in our pipeline, and this seems to work for us.
steps:
  - task: NuGetAuthenticate@0
    displayName: "Authenticate with NuGet"
    inputs:
      forceReinstallCredentialProvider: true

  - task: UseDotNet@2
    displayName: Install .NET Core
    inputs:
      useGlobalJson: true

  - task: Cache@2
    displayName: Cache NuGet packages
    inputs:
      key: "nuget | $(Agent.OS) | **/global.json, **/*.csproj"
      restoreKeys: "nuget | $(Agent.OS)"
      path: $(NUGET_PACKAGES)

Usage example:
jobs:
  - job: Compile
    displayName: "⚙ Compile"
    dependsOn: []
    steps:
      - template: setup-build-task.azure-pipelines.yml

      - task: CmdLine@2
        displayName: Restore, lint, and build
        inputs:
          script: "./build.cmd RestoreTools Restore Lint Compile --skip"

Godspeed.

For the record, since this issue was closed I created this issue in the developer community: https://developercommunity.visualstudio.com/t/dotnet-tool-installrestoreupdate-not-w/10455207


WIP: Github Actions

Pull Request Test Coverage Report for Build 254

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 99.292%

Totals

Change from base Build 253:
0.0%

Covered Lines:
421

Relevant Lines:
424

💛  - Coveralls


Migrate headerOverlay string ref to functional ones in channel_info_button.js

Fixes https://github.com/mattermost/mattermost-server/issues/15797#event-3844313554
JIRA ticket: https://mattermost.atlassian.net/browse/MM-29308

Hello @sowmiyamuthuraman,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

/update-branch

/update-branch

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.

Trying to auto merge this PR.

Pull Request successfully merged
SHA: fbc5157918a0b9a366b77fac0b1cbd4d78a217c6


make create just call get instead

Description of Change
Instead of returning a noop 200 response, we could return the response from get(). That way, it will contain the same response data as before.
One issue with this is that getting a detection list profile by username is more delayed than getting a profile by ID, for newly created users. However, anybody's script that was calling create for newly created users would have already been failing from the API, so this is fine imo.
Issues Resolved
n/a
Testing Procedure
Call the method sdk.detectionlists.create_user() and make sure the response is 200 for users that exist.
PR Checklist
Did you remember to do the below?

[x] Add unit tests to verify this change
[x] Add an entry to CHANGELOG.md describing this change
[x] Add docstrings for any new public parameters / methods / classes

Should we also delete the create method from the user_profile_service?

Also, what about having a custom exception raised if the user isn't found in the create call, as that could be confusing if someone tries to create a profile for a non-existent and/or typo'd username and just gets a Could not find user error response.

Should we also delete the create method from the user_profile_service?

I don't think that service gets exposed directly by the SDK object in any way, so that should be safe


An option to send unspent amount to the same origin address

I am writing a service that will make small transactions from the same wallet/address using cardano-wallet api. Initially, I have funds only on the one address. But once I make a small transaction, the unspent amount is sent to the next unused address. So after 10k TXs cardano-wallet is monitoring 10k addresses for incoming TXs. That slows it down and makes it impossible to use after the TX number grows.
It would be a useful feature if you could add a non-required option like unspent: "origin" that will send the unspent amount to the same origin address instead of sending it to the next unused address.
The same problem happens with the pool's pledge. Once you send any small amount from your pledge address, all funds will move to another unused address, and the pledge will not be met. It leads to huge problems with missing the whole epoch rewards once you forget about that behavior and do not make an additional TX for all wallet funds to the original address.

Missed epoch rewards as well (248), after registering pledge wallet in voting (actual transaction). As a temporary solution I sent all funds back to the original address.

https://adapools.org/pool/cc3bf31353e971540a284bb7da30cfaac4b0d4138e03bbdf85013eca
3 blocks minted, but no rewards for epoch 248 despite of meeting pledge the whole epoch (base address was fine)

Thanks for your feedback on this. This is certainly something we've been considering for a while but hasn't got any major priority yet.

That slows it down and makes it impossible to use after the TX number grows.

Do you have any numbers backing this claim?

Do you have any numbers backing this claim?

Yes. I do. I will attach some screenshots and errors I am getting performing the same actions on a wallet with used 6.3k addresses vs 50.
At the moment I was making those screenshots, cardano-wallet was not sending or making any tx, so the comparison should be correct. I left it just running without performing any actions so Load average could be compared.
This is the server load with 50 addresses:

Here the same server, same wallet, idle with 6.3k used addresses.

Compare "Load average". And it's just 6.3k addresses, not like 50k or 100k. I will do the testing with it, but there are some serious problems with cardano-wallet that do not allow me to get to that number.
Ok. Next, let's take a look at the latest Daedalus that used cardano-wallet.
50 addresses:

and it's memory consumption:

And now 6.3k addresses:

and the memory used 😱

😱😱😱😱😱
Now let's move to another problems that makes cardano-wallet unstable and unusable... I got to 6.3k used addresses by running tiny transactions to the same 1st address. I sent api requests in loop with the same size, but different metadata, so I could distinguish TXs. API server was working quite stable but when it got to ~5-6k used addresses it started throwing errors out of nothing like:
    [code] => not_enough_money
    [message] => I can't process this payment as there are not enough funds available in the wallet. I am only missing: coin: 1.000000 tokens: []

although the wallet had a couple of thousand ADA on it.
And:
    [code] => cannot_cover_fee
    [message] => I am unable to finalize the transaction, as there is not enough ada I can use to pay for fees, or to satisfy the minimum ada quantities of change outputs. I need about pretty 1.171089 ada to proceed; try increasing your wallet balance as such, or try sending a different, smaller payment.

And even:
    [code] => created_invalid_transaction
    [message] => That's embarrassing. It looks like I've created an invalid transaction that could not be parsed by the node. Here's an error message that may help with debugging: HardForkApplyTxErrFromEra S (S (S (Z (WrapApplyTxErr {unwrapApplyTxErr = ApplyTxError [LedgerFailure (UtxowFailure (UtxoFailure (BadInputsUTxO (fromList [TxInCompact (TxId {_unTxId = "e319e62b3e1419b870cbd4edf37135ca91325149b6463512105b7a2ce0dabe67"}) 1])))),LedgerFailure (UtxowFailure (UtxoFailure (ValueNotConservedUTxO (Value 2000000 (fromList [])) (Value<PHONE_NUMBER> (fromList [])))))]}))))

The more addresses were used, the more often it was throwing such errors and when I finished testing getting to 6.3k used addresses, it was  around 90% of time that API server threw such errors instead of making a TX. Before that, going up to 5k I have almost 0% rate of any problems.
Cardano-wallet started having problems like:
Feb 26 16:54:56 web bash[252081]: SQLite3 returned ErrorConstraint while attempting to perform step: UNIQUE constraint failed: seq_state_pending.wallet_id, seq_state_pending.pending_ix
and
Feb 26 16:55:57 web bash[252081]: [cardano-wallet.api-server:Error:47810] [2021-02-26 13:55:57.51 UTC] [RequestId 24520] 500 Internal Server Error in 1.975678921s
And that was only 6.3k used addresses, not like 50k or 100k. I guess it will just die on such numbers. I will try to get there, but it's not running smoothly already and fails.
So I guess the issue is there and using new addresses for the unspent amount may be an illusory choice for keeping privacy (it's really not because it can all be tracked back to the original address) but it's making more problems than any good.
I really hope that this post will receive more attention from contributors and an option for using less addresses will be implemented.
PS: cardano-wallet is still my choice to work with, although atm I will probably have to switch wallets occasionally with moving all funds to a new a wallet and deleting the old one. Not  a good solution, but at least something.

Seed to this testnet wallet is:
goddess jump bitter excite sugar desk book spend tiger quantum smooth donor input bacon naive midnight pulse isolate forward mask drama little wear romance
Only rare tries to make a TX will go through. cardano-wallet even thinks that it has 0 balance most of the time and fails with not_enough_money a lot. While testing it I also detected another bugs related to Daedalus. I will open an issue there.

@vicepool thanks for the detailed reports, that is useful. Some numbers above aren't right. I do have a few extra questions to understand the problem a bit better.

I reckon your wallet is a Shelley wallet (i.e. created via the /v2/wallets endpoint)  ?

 At what pace? Why am I asking this is because, in order to make transactions, you need to have available UTxO on your wallet. When you send a transaction, its associated UTxOs are locked until the transaction is inserted and seen in the ledger. So, even if your total balance is large, your available may not be, resulting in the error you've reported above.

From the error message above, I can tell that you haven't been using any of the two latest releases. So, what release are you using?

using new addresses for the unspent amount may be an illusory choice for keeping privacy (it's really not because it can all be tracked back to the original address) but it's making more problems than any good.

Yes. Unfortunately, the current scheme for delegation makes the whole privacy argument pointless. However, using new addresses shouldn't be a major problem for most users. We've exchanges with 100k+ addresses in their wallet working correctly. Although they're using a different type of wallet, I am a bit surprised about your report but be sure we'll address it nonetheless.

@vicepool I've just tried restoring your wallet. It's doing quite fine on my side. The wallet contains ~6.7K addresses, ~16.8K transactions. The CPU usage once restored is basically close to 0 and constant and it demands about 1.3GB of RAM (that sounds a little bit high, but nothing quite alarming at this stage).
I've tried estimating fees and sending a transaction, it went through pretty much instantaneously. May I ask an extra question:

How did you start the wallet (what command, options and runtime flags) ?

* How did you start the wallet (what command, options and runtime flags) ?

It runs as a daemon with:
cardano-wallet serve --testnet /home/cardano/cardano-node/testnet-byron-genesis.json --node-socket /home/cardano/cardano-node/db/socket --database /home/cardano/cardano-node/wallets --log-level notice

Fixed, one_change_address_mode in the create wallet endpoint.


pypy 启动报错

pypy 安装成功，但在import dpark的时候报错
root@test:/opt# pypy
Python 3.5.3 (3f6eaa010fce, Jan 11 2018, 04:44:35)
[PyPy 5.10.1 with GCC 6.2.0 20160901] on linux
Type "help", "copyright", "credits" or "license" for more information.

from dpark import DparkContext
Traceback (most recent call last):
File "", line 1, in
ImportError: cannot import name 'DparkContext'

Can you provide some more infomations?
For example, you may try this:
import dpark
print(dpark.__path__)
print(dir(dpark))

.. fix
Not fully installed....


Namespace for referenced element was wrong.

Hallo,
I found a bug with a wrong namespace for a referenced element. I think this only occurs under special circumstances - please have a look at the unit test that I've included. The bug was that the element "name2" didn't get the namespace prefix "cis:" and therefore the XML file was invalid.
There is still another bug in the library, though:
The "streetName"-node should also have the "cis:"-prefix, but that one is also omitted. That's the reason why the unit test still fails.
I tried to find out the cause of this bug but haven't been successful so far. I'll try again tomorrow, but if anybody has an idea where I have to look please give me a note.

Coverage decreased (-0.05%) to 93.596% when pulling 5450882c7bda300556fafbd5803c5c432c8d9b91 on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

Coverage decreased (-0.05%) to 93.596% when pulling 5450882c7bda300556fafbd5803c5c432c8d9b91 on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

Coverage decreased (-0.05%) to 93.596% when pulling 5450882c7bda300556fafbd5803c5c432c8d9b91 on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

@blissi this is great!  Looks like the test is failing.  Looks like elements names are different in the schema?

Coverage decreased (-0.05%) to 93.596% when pulling bb65e1af09679f3c00a7c82deee7361819f5f4d0 on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

Coverage decreased (-0.05%) to 93.596% when pulling bb65e1af09679f3c00a7c82deee7361819f5f4d0 on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

Have a look at my unit test update I just checked in:
I split the test into two tests:

doAction__should_handle_namespaces_of_referenced_element_correctly: here the bug in the library was that name2 was not prefixed with "cis:" => I have fixed this bug in my previous checkin.

doAction__should_handle_namespaces_in_referenced_complex_type_correctly: this is the other bug I'm still struggling with. streetName is a member of the complexType "PersonAddressType" and should also have the prefix "cis:".

Hmm, it looks like you're using element names that aren't defined in your schema.  Before your last update, cis:name appeared in the failed assertion, now only name appears in the assertion.

@jsdevel I don't fully understand...in the new unit test,
<name>My name</name>
is correct since it is defined under  and thus has the namespace bcs. It's the streetName that isn't correct => the "cis:"-prefix is missing in the generated request and therefore it also mistakenly inherits the bcs-namespace.

  Build output used cis namespace for the name elements.  Now you're just defining type xs:string for name and streetName.  What does the spec say about namespace behavior?

Yes, I changed it when adjusting the unit tests for the second checkin. I did it on purpose:
I wanted to see in the result-XML of the unit test that the name below <Person> has the correct namespace (omitted here because inherited). The name below <Address> on the other hand must have the namespace. Sorry for the confusion.
I don't know where this is officially specified. I found this bug because the webservice I am calling is rejecting the request. When I specify the namespace prefix, it works. The webservice I am calling seems to behave correctly: when I use the WSDL in SoapUI it also creates a sample request where the "cis:"-prefix is used at the name below <Address>.
I think I am quite close to a solution...I have to dig deeper in the evening and try some more things. I'll keep you informed.

@jsdevel  The bug is fixed now.

Coverage decreased (-0.01%) to 93.634% when pulling 4627df3ae2cbd7d72edc3044f224c9bafe2cc82a on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

My last fix had a severe regression: when one SOAP operation was called twice, the second call failed because I had manipulated the $name and therefore findChildSchemaObject() couldn't find that schema object anymore.

Coverage decreased (-0.01%) to 93.634% when pulling f160b4366801a71f3d857747e1a79996f62f4f9f on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

Coverage decreased (-0.01%) to 93.634% when pulling f160b4366801a71f3d857747e1a79996f62f4f9f on blissi:master into a142aee54fac933d75269defcdc6b62708fca328 on vpulim:master.

Thanks a lot for your contriubtion @blissi - could you please rebase against the latest master as I updated our .travis.yaml so that the latest LTS version of node is used for testing

@blissi any chance you can clean this PR up?

ping @blissi - if you could take some time to clean up and rebase your PR we'll happily merge it 👍 :trollface:

@herom  I'm not a git expert - how do I do this?

@blissi just take a look into our section on Submitting a Pull Request of our (CONTRIBUTING.md)[https://github.com/vpulim/node-soap/blob/master/CONTRIBUTING.md] file. There are 2 links to the (very good) GitHub Help Pages on this topic.

This pull request is still not fixed ?

closing due to inactivity


Remove public packet fields

Removes public access to struct fields on the following items:

[x] src/packet/ack.rs
[x] src/packet/error.rs
[x] src/packet/mod.rs
[x] src/packet/data.rs
[x] src/packet/rq/wrq.rs
[x] src/packet/rq/mod.rs
[x] src/packet/rq/rrq.rs

I'm really sorry for the toil, @itmecho, but after reading a bit about some more Rust idioms I'm not sure we should add accessor methods. It seems the general consensus would be to add them to protect invariants or if certain combinations of fields don't make sense. However, we don't really have any invariants to protect here and I don't think the packet module should impose any additional policies about the data it manages.
What are your thoughts?

No worries! Do you have any links so I can read up on it as well?

Yeah! The main resource I've been reading is the Rust API Guidelines and also seeing what others thing on Reddit posts like this one.
Sounds good. I will go ahead and close the issue I opened.
Thanks again (and sorry for the work)!

Closing per discussion above.

@connorkuehl Thanks for those links! Super useful =]