Feature/12/confirm block hashes

This changes should give response based on state in db
#12
IMPLEMENTATION NOTES

Orchestrator registers consensus service after pandora and vanguard stopped flooding with pending headers. This is done to prevent crawling on top of incomplete data
Consensus service aims to crawl from latest verified slot (0 when new) on top up to latest valid verified pair. By valid ATM I treat that header was present on pandora and vanguard database in particular slot. Database is extended with batchLimit so it fetches batch from each database, for example fromSlot 6, bachlmit 500 -> it will give range from 6 to 506. This is done because next pending data can come into each database so I limit it due to data availability and sustainability
crawler.go is written in functional approach to limit abstraction to minimum. During the research I have found out that there can be at least 3 sorting scenaraios that must happen after you fully release canonicalness. Functions in this file are all private because it was only code separation. They should be invoked in particular order. Responsibility of keepieng things done in order  are stored in function called Canonicalize
Canonicalize should be singleton, it means that during lifecylcle it should be run once at a time. It uses raw transaction pattern within when aggegated data must be inserted into databases. Altough there is no rollback ATM when something goes wrong. Idea is that you recrawl again after something bad happens.
Due to the fact that we use 3 buckets at once (pandora, vanguard, realm) to resolve canonicalness all crucial logic is held in non blocking select and will return anytime it encounter an error on any of sides. Due to my understanding of golang I assume that execution on default case will be stopped and any of goroutines that were invoked there will be able to finish execution. I used this pattern to prevent defer aggregation. LIFO can be tricky, so early return and short circuits are more handy for that case
There are various of things you need to do until you resolve states PENDING, VERIFIED, SKIPPED. I tried to do things in memory and save to database only when needed. Mutex should prevent data concurency

verified is the easiest one. You need to have proper data on both sides. I chose vanguard as the leader in that case
pending is the next to resolve. if we have highest verified slot then all higher slots are pending in that range
skipped is quite tricky. I decided to resolve skipped in 3 ways. First of all get all orphans below highest verified slot (present only on 1 side) and mark them as skipped, After all I take data from database on same range and get all nil entries to also mark them as skipped. I use map[uint64]*RealmPair{} to not duplicate entries and push all into []*RealmPair

Thank you for your comments @atif-konasl

@blazejkrzak IMHO this implementation looks very complex and resource intensive. You have used 10+ go routines and many db calls for solving this issue whereas we could solve this issue with 1 or 2 go routines and less db calls. I think, we need to refactor this implementation in future.
For now this implementation is OK for me. After finishing our all implementation we must need to refactor this implementation.

I would love to know how to do it with 1 or 2 routines

On top of this changes setup works:
Any new changes on top of this setup should behave almost the same and not be regressive.
Staging:
- VANGUARD_GH_TAG: v0.0.14-zeta
- PANDORA_GH_TAG: p-v0.0.10-alpha-interval-gh-actions
- ORCHESTRATOR_GH_TAG: v0.0.24-alpha


fix(Anchor): Anchor focus colors

Overview

PR Checklist

[ ] Related to designs:
[ ] Related to JIRA ticket: [ABC-123]
[ ] I have run this code to verify it works
[ ] This PR includes unit tests for the code change
[ ] This PR includes testing instructions tests for the code change
[ ] The alpha package of this PR is passing end-to-end tests in all relevant Codecademy repositories

Testing Instructions

Don't make me tap the sign.
PR Links and Envs

Repository
PR Link
PR Env

Monolith
Monolith PR
Monolith Env

Portal
Portal Link
Portal Env

Another Repo
Another Link
Another Env

📬Published Alpha Packages:@codecademy/gamut@55.7.2-alpha.08e344.0@codecademy/gamut-kit@0.6.351-alpha.08e344.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://651c6eb2d736b61349eb7e3b--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.2-alpha.49e5df.0@codecademy/gamut-kit@0.6.351-alpha.49e5df.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://651c823ed7f20b1f46cd2488--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.3-alpha.875d95.0@codecademy/gamut-kit@0.6.352-alpha.875d95.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://651d75cc096c0b006ebecf00--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.3-alpha.6d7f24.0@codecademy/gamut-kit@0.6.352-alpha.6d7f24.0<EMAIL_ADDRESS>
📬Published Alpha Packages:@codecademy/gamut@55.7.4-alpha.02528b.0@codecademy/gamut-kit@0.6.353-alpha.02528b.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://651f1d0a03c8cf02bb67c1ae--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.4-alpha.65ad79.0@codecademy/gamut-kit@0.6.353-alpha.65ad79.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://652037913634d810fb04ffc1--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.6-alpha.d0d740.0@codecademy/gamut-kit@0.6.355-alpha.d0d740.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6526aefa2cf8ce71b74a11a9--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.6-alpha.67e521.0@codecademy/gamut-kit@0.6.355-alpha.67e521.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6526b39bf1746830aadb80f3--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.6-alpha.bc240e.0@codecademy/gamut-kit@0.6.355-alpha.bc240e.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://6526c78a7d422a0c5f5c9d82--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut@55.7.6-alpha.87e5f8.0@codecademy/gamut-kit@0.6.355-alpha.87e5f8.0@codecademy/styleguide@66.5.7-alpha.87e5f8.0


Point vs point confusion

'Point' is an classy object that has a position, projections and some useful methods; a 'point', mathematically, can be a list of integers/floats or a numpy array. This causes massive confusion in code and tests.
The bug fixed in feature/offset is the result of this confusion, there are probably more.
TODO:

Change names of 'point' variables (not objects) to 'position' or similar
Refactor test_vertex_list and test_edge_list with better shortcuts
Tests of the new Vertex.from_point() functionality

Renamed 'point' to 'position'.


scale down arangodb deployment for dev environments

My Environment

ArangoDB Version:        3.9.2
Deployment Mode:       Cluster
Deployment Strategy:   Kubernetes
Infrastructure:               GKE

deactivating (Scale down) of Arangodb cluster spawned with the operator via a CRD
Hello,
I am spawning a lot of dev environment with the latest arangodb operator (CRD yaml below).
I was wondering if there is an easy way to scale the cluster at night time / weekend to save cost.
Since the operator creates pod directly, without a managing object such as deployment or statefulset, seems like my only is to update the CRD with count 0 for all components (agent, coordinators, and database servers).
However, when doing so the operator seems stuck and unable to perform this action.
Please advise.
CRD for reference:
apiVersion: database.arangodb.com/v1
kind: ArangoDeployment
metadata:
  name: arangodb-cluster
  namespace: infip-test
spec:
  agents:
    count: 3
    nodeSelector:
      cloud.google.com/gke-nodepool: default-pool
    overrideDetectedTotalMemory: true
    resources:
      limits:
        cpu: 200m
        memory: 200Mi
      requests:
        cpu: 200m
        memory: 200Mi
  auth:
    jwtSecretName: None
  coordinators:
    count: 1
    nodeSelector:
      cloud.google.com/gke-nodepool: default-pool
    overrideDetectedTotalMemory: true
    resources:
      limits:
        cpu: 200m
        memory: 200Mi
      requests:
        cpu: 200m
        memory: 200Mi
  dbservers:
    count: 2
    nodeSelector:
      cloud.google.com/gke-nodepool: default-pool
    overrideDetectedTotalMemory: true
    resources:
      limits:
        cpu: 1
        memory: 500Mi
      requests:
        cpu: 1
        memory: 500Mi
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: ssd
  disableIPv6: true
  downtimeAllowed: false
  environment: Development
  externalAccess:
    type: None
  image: arangodb/arangodb:3.8.6
  memberPropagationMode: always
  metrics:
    enabled: false
  mode: Cluster
  networkAttachedVolumes: true
  tls:
    caSecretName: None

Hi, Would like to know the solution for that as well. We also scale down the environments for the night and seems not to have a good strategy for that.


Figure out if AngularController.java is required

AngularController.java seems to be necessary in order to workaround the following issue after integrating Stormpath and hitting http://localhost:8080.
2016-10-06 14:14:03.359 DEBUG 16973 --- [  XNIO-2 task-1] enticationSpringSecurityProcessingFilter : No token in request, will continue chain.
2016-10-06 14:14:03.380 DEBUG 16973 --- [  XNIO-2 task-1] .s.s.c.StormpathAuthenticationEntryPoint : Pre-authenticated entry point called. Rejecting access
2016-10-06 14:14:03.385 ERROR 16973 --- [  XNIO-2 task-1] .s.s.c.StormpathAuthenticationEntryPoint : Couldn't resolve content type

com.stormpath.sdk.servlet.http.UnresolvedMediaTypeException: No MediaType could be resolved
for this request (text/html,application/xhtml+xml,image/webp,application/xml;q=0.9,*/*;q=0.8) and
the configured producesMediaTypes settings (application/json). The ContentNegotiationResolver
was not able to come up with a valid MediaType.

Fixed by adding AngularController.java from spring-boot-webmvc-angular. After doing this, when I hit localhost:8080, it redirects to http://localhost:8080/login?next=%2F#/. I was able to fix this redirect issue by adding .antMatchers("/**").permitAll() to JHipster’s SecurityConfiguration.java.

It needs to be removed when using the JHipster UI (see #7), but it is required when you integrate the Angular example in a JHipster-generated application.


Compliance Report B | Total Debits |2|

Story:
As a TFRS user, I want to see the total debits incurred from all of the fuel I am reporting as being supplied within Schedule B - Part 3 Fuel Supply.
Task:

Within the Schedule B sub-tab, show the total debits incurred from all of the fuel that is reported within this schedule.
The total debits would be determined by adding up all of the negative values (and only the negative values) from the credit or debit column (Card 1397). Values in this column that are positive (credits) should be ignored in calculating the total debits incurred.

Design Requirements:

The value should be displayed as a positive, whole number
The corresponding value should be displayed in the format that places a comma every 3 digits. We are dealing with large volumes and unformatted number values can be difficult to read at a glance. For example, instead of 4327423 have it display on the screen as 4,327,423.

Potential Design(s):

Within the existing excel-based compliance report, the total credits and total debits are displayed near the bottom of the page (below the reporting of the individual fuels supplied; see attached screenshot). This location could be maintained or the values could be placed in a more prominent location (e.g. near the top of the page).
While the resulting value will likely not be a whole number (will contain a decimal/fraction of a credit/debit), the value should be displayed as a whole number (no decimal / fraction of a credit/debit) and use conventional rounding (1-4 round down, 5-9 round up) when displaying. Note: this is just a display requirement; we will want to maintain the decimal/fraction values when we total up the credits and debits (we just don't want to display them). https://trello.com/c/j2PDBc2f/1406-compliance-report-b-total-debits-2

 Compliance Report B | Total Debits |2|


[subscriptions] need an order removed from Awaiting payments

need order # 907194 removed from Awaiting Payments in Subscriptions. They are getting a free 3 yr instead which is already taken care of. Thanks

This is surely unrelated to this project.


From filter with 3 lists cannot resolve refernece to the 3rd item

The from filter shown below worked in CQL 1.2.57 and returns an error in CQL 1.3.10.
Filter
define "Dates of Well-Care Procedure Visits With PCP During First 15 Months of Life":
from "Well-Care Procedures During First 15 Months of Life" WellCareProceduresDuringFirst15Months,
"Primary Care Practitioners" PrimaryCarePractitioner,
"Encounters During First 15 Months of Life" EncountersDuringFirst15Months
where
(
exists (
EncountersDuringFirst15Months.diagnosis.condition Condition
where EndsWith(Condition.reference, WellCareProceduresDuringFirst15Months.id)
)
and
(
exists (
EncountersDuringFirst15Months.participant.individual Individual
where EndsWith(Individual.reference, PrimaryCarePractitioner.id)
)
or  exists (
WellCareProceduresDuringFirst15Months.performer.actor Performer
where EndsWith(Performer.reference, PrimaryCarePractitioner.id)
)
)
)
return ToDate(WellCareProceduresDuringFirst15Months.performed."start")
Error: Could not resolve variable reference EncountersDuringFirst15Months

Resolved by #220


Can not paste a path to search in mini.pick

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md
[X] I have updated 'mini.nvim' to latest version

Module(s)
mini.pick
Description
Can not paste a path to search in mini.pick (Pick File)
Neovim version
0.9.5
Steps to reproduce

Start nvim (already installed mini.pick plugin using stable version)
Open minipick window (using MiniPick.buildtin.files)
Cmd + v to paste a path I copied.
Refer below attachment to see the issue:

Expected behavior
It should paste my path into mini.pick textbox and trigger searching for that text
Actual behavior
It does nothing

Thanks @echasnovski


Minimum system requirements

So I just discovered this project, but I plan on implementing this on a very low powered ("performance") microcontroller.
Is 2 x 300 MHz 32bit with 1 megabyte ROM & 512 kilobyte RAM enough?
Could I squeeze it down even further. I'd need it to be real time too.
Any benchmarking or related projects I could look in to, thank you.

Please use Tesseract users forum and ask this question (and other questions you might have) there.


Where can I change the setting for my process iteration duration?

If I am using a Scrum based process for my devops project, where can I alter or customize the iteration duration for my Sprint in Azure DevOps Service?
I was reviewing the demo labs for Azure DevOps services.
I noticed that when I tried to create a new iteration (sprint), It was selecting 3 weeks by default based on sample examples of boards in PartsUnlimited project sample.
So is it automatically detecting the sprint duration based on existing (previous) sprints?
Or is there a setting somewhere which I can edit to set the default duration of iteration in my DevOps Project?
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 715eca62-8a31-4163-c27b-f5e71d781982
Version Independent ID: 06dc711f-1946-29e9-aece-75af903da09b
Content: Customize the workflow of an inherited process - Azure DevOps Services
Content Source: docs/organizations/settings/work/customize-process-workflow.md
Product: devops
Technology: devops-agile
GitHub Login: @KathrynEE
Microsoft Alias: kaelli

@Anand-Moghe - You specify the duration of a sprint by setting the Start and End date of the sprint. Described here: Define iteration paths (aka sprints) and configure team iterations. The system detects the pattern you set when you define the dates and prompts the start and end dates for the next sprint you define - however, you can override that and change it at any time.


Update python version 1.5.0

Docs Build status updates of commit 8dea48e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

prosepy/docs-ref-conceptual/knownissues.md
:white_check_mark:Succeeded

prosepy/docs-ref-conceptual/relnotes.md
:white_check_mark:Succeeded

xrefmap.yml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Escaping % and _ in like SQL queries

Describe the feature
Allow escaping % and _ in like SQL queries sent to database using ESCAPE clause as described on:
https://www.sqlitetutorial.net/sqlite-like/
https://mariadb.com/kb/en/like/
https://www.postgresql.org/docs/current/functions-matching.html#FUNCTIONS-LIKE
Motivation
According to  https://github.com/go-gorm/gorm/issues/5828#issuecomment-1377642307 GORM doesn't allow searching using literal % nor _ in SQL like queries.
Related Issues
https://github.com/go-gorm/gorm/issues/5828

According to https://github.com/go-gorm/gorm/issues/5828#issuecomment-1377642307 GORM doesn't allow searching using literal % nor _ in SQL like queries.

Gorm currently does not support Like api, but we support custom Clauses, you can implement it yourself, we will be happy if you are willing to create a package or pr for it.
Here is a simple implementation
  word := "%"
  query := escapeLike("%", "%", word)

func escapeLike(left, right, word string) string {
    var n int
    for i := range word {
        if c := word[i]; c == '%' || c == '_' || c == '\\' {
            n++
        }
    }
    // No characters to escape.
    if n == 0 {
        return word
    }
    var b strings.Builder
    b.Grow(len(word) + n)
    for _, c := range word {
        if c == '%' || c == '_' || c == '\\' {
            b.WriteByte('\\')
        }
        b.WriteRune(c)
    }
    return left + b.String() + right
}

According to #5828 (comment) GORM doesn't allow searching using literal % nor _ in SQL like queries.

Gorm currently does not support Like api, but we support custom Clauses, you can implement it yourself, we will be happy if you are willing to create a package or pr for it.
Here is a simple implementation
  word := "%"
  query := escapeLike("%", "%", word)

func escapeLike(left, right, word string) string {
    var n int
    for i := range word {
        if c := word[i]; c == '%' || c == '_' || c == '\\' {
            n++
        }
    }
    // No characters to escape.
    if n == 0 {
        return left + word + right
    }
    var b strings.Builder
    b.Grow(len(word) + n)
    for _, c := range word {
        if c == '%' || c == '_' || c == '\\' {
            b.WriteByte('\\')
        }
        b.WriteRune(c)
    }
    return left + b.String() + right
}

is not work . i think it need to use MySQL . like SELECT * FROM operation_log WHERE content LIKE '%\%' ESCAPE ''; this is work


fix(build/#3769) Fix npm watch command

Fixes #3769
To be honest I'm not sure whether it was broken or just didn't work on my pc. Hoping it's the first scenario, I've decided to submit a PR.
Also,  should this PR be included in CHANGES_CURRENT.md?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: bryphe:x: eEQKYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the PR @eEQK !

To be honest I'm not sure whether it was broken or just didn't work on my pc. Hoping it's the first scenario, I've decided to submit a PR.

Nice catch, it actually was broken. Missed updating the command when bringing over tree-sitter & vterm (ie, #3555 ). Appreciate the fix!

Also, should this PR be included in CHANGES_CURRENT.md?

For sure, we have an Infrastructure section for build / CI related improvements. Just added it 👍


isProvider() not working as expected

Consider a provider check in a hook:
iff(isProvider('external'), doSomethingForExternal())
Expected behavior
doSomethingForExternal() is executed only if context.params.provider === 'external'
Actual behavior
doSomethingForExternal() is executed if context.params.provider === 'rest'
Replacing isProvider() works as expected:
iff(context => context.params.provider === 'external', doSomethingForExternal())
Module versions v6.1.0
Possibly an issue with the comparison in index.js - line 784:
providers.some((provider) => provider === hookProvider || provider === "server" && !hookProvider || provider === "external" && !!hookProvider)

It's not a bug, it's a feature ™
How does it come, that you use context.params.provider as 'external'? I don't know any feathers library, that does this.
isProvider expects your input as: 'socketio' | 'rest' | 'primus' | undefined. That are the common values for context.params.provider. If you set it yourself, please set it to socketio or rest to be consistent with other libraries.


typespec-validation structure rule doesn't seem to detect if the main.tsp is in resource-manger folder

From this PR https://github.com/Azure/azure-rest-api-specs-pr/pull/16294 which had a bad structure(existing) it wasn't complaing
/specifications/provider/resource-manager/main.tsp instead of
/specifications/provider/MyService.Management/main.tsp
Guessing we can see the same problem with data-plane folder.
Maybe we need to just add some explicit check to make sure the typespec wasn't created under those. Problem is right now avocado will then complain that you have unreferenced examples because it think the typespec examples are mean to be referenced from a swagger file.

This should have been caught here:
https://github.com/Azure/azure-rest-api-specs/blob/8290f26b5a481b37c0e098593e475a9321a0cfe3/eng/tools/typespec-validation/src/rules/folder-structure.ts#L49-L56
Folder resource-manager should have failed our capitalization check.  But we must have a bug.  I will add a test and fix the bug.

Verified fixed:
https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3343503&view=logs&j=52599800-cfcd-5615-dd98-137d26e8cb9a&t=c0acbdc7-6ccc-5699-6c3a-c41428f73fe2&l=56


Sometimes ending a Holographic Remoting session generates exceptions

Describe the bug
Sometimes when ending a Holographic Remoting session, one or more exceptions can be generated. This can happen with both XR SDK and OpenXR pipelines. I found this issue while quickly starting and stopping remoting sessions to switch between the Hand Menu Examples scene and the Hand Interaction Examples scene. The XR SDK pipeline has different exceptions that get thrown than the OpenXR pipeline. The warnings and exceptions are not always thrown when stopping a remoting session, there appears to be some sort of magic timing needed. This issue either does not happen in Unity 2019 LTS or is much more difficult to trigger than the Unity 2020 LTS.
To reproduce
Steps to reproduce the behavior:

Create a new project, import the MRTK, and enable the XR SDK or OpenXR pipeline
Open either the Hand Interaction Examples scene or the Hand Menu Examples scene
Change to the 'DefaultHoloLens2XRSDKConfigurationProfile' for XR SDK or 'DefaultOpenXRConfigurationProfile' for OpenXR
Configure Holographic Remoting in Unity for your device
Press the 'Connect' button to connect to the device
Press the 'Play' button to start the remoting session and wait for the scene to start rendering on device
Press the 'Play' button stop the remoting session.
Open the other scene that was not chosen in step 2
Press the 'Connect' button to connect to the device
Press the 'Play' button to start the remoting session and wait for the scene to start rendering on device
Press the 'Play' button stop the remoting session.
Check the Unity console for errors
If there are no unexpected behaviors, repeat these steps

Screenshots
XR SDK:

OpenXR:

Your setup (please complete the following information)

Unity Version 2020.3.2f1
MRTK Version 2.6.1

Target platform (please complete the following information)

HoloLens 2
UWP
OpenXR
XR SDK

Additional context
Add any other context about the problem here.

Possibly related to https://github.com/microsoft/MixedRealityToolkit-Unity/issues/8459

Exceptions fixed by https://github.com/microsoft/MixedRealityToolkit-Unity/pull/9868


is Propel 1.x is still maintained?

last commit was about  a year ago, and the issue are piling up, but no active development seams to be done.
So is propel 1 not maintained anymore ?

Yes, nobody has obviously time to fix all those issues. also interesting http://propelorm.org/blog/2015/06/27/propel2-current-state-and-future.html


Veltron exchange

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5a34424e-6a23-5517-5755-c9282643876f
Version Independent ID: 1c375649-94d6-863a-203b-cb5ed5e4044e
Content: Continuous deployment for Azure Functions
Content Source: articles/azure-functions/functions-continuous-deployment.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@vechiverme2050 Do you have any feedback to share? Looks like you've forgot to add feedback. Please let us know your query.

@vechiverme2050 Do you have any feedback to share? Looks like you've forgot to add feedback. Please let us know your query.

We try to reserve these issues for feedback directly related to the tutorials: like if there are errors that block tutorial completion or if there are updates/changes required, suggestions for improvement, etc. if you have a product-related question, please post to Microsoft Q&A and we will happily continue the conversation.
Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.

We try to reserve these issues for feedback directly related to the tutorials: like if there are errors that block tutorial completion or if there are updates/changes required, suggestions for improvement, etc. if you have a product-related question, please post to Microsoft Q&A and we will happily continue the conversation.
Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.


i cant install extensions

Issue Type: Bug
i cant enable any extenions and i cant run any code and when i open any extenion gets an error any body help?
VS Code version: Code 1.47.0 (d5e9aa0227e057a60c82568bf31c04730dc15dcd, 2020-07-09T08:02:06.629Z)
OS version: Windows_NT x64 10.0.18363

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz (8 x 1800)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: enabledrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
7.92GB (1.65GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (4)

Extension
Author (truncated)
Version

aio-compiler
azt
2.0.0

code-runner
for
0.10.0

python
ms-
2020.4.76186

cpptools
ms-
0.29.0-insiders2

Please post more details. What happens that shouldn't? What doesn't happen that should? Did the problem only appear after 1.47 arrived? If so, were you previously on 1.46.1?

I have the same problem, all plugins failed to start. And it only happened in version 1.47.

I have the same problem, just loading but no result.

Please follow these steps

Help -> Toggle Developer Tools -> Network
Then check for any network failed requests while browsing and installing the extensions if found all requests fails then its probably a proxy or DNS issue at your end.
To confirm its a DNS issue check if you can access this page https://ms-python.gallerycdn.vsassets.io/extensions/ms-python/python/2019.8.29288/1565127378610/Microsoft.VisualStudio.Services.Icons.Default, if you cannot please try configuring your DNS like pointing to <IP_ADDRESS> as suggested here? https://<IP_ADDRESS>/dns/. Otherwise please contact your network administrator.
If it is not a network issue, please share the logs from the following locations

Help -> Toggle Developer Tools -> Console
View -> Command Pallette... -> Toggle Shared Process -> Console
View -> Open View… -> Log (Shared)
View -> Open View… -> Log (Window)

Please confirm that if you are having this issue - https://github.com/microsoft/vscode/issues/102082

after too many tries and after the update
I changed the DNS protocol to google's protocol and it worked
https://developers.google.com/speed/public-dns
the V4 and V6
On Tue, Jul 21, 2020 at 1:23 PM github-actions[bot] <
<EMAIL_ADDRESS>wrote:

Closed #102052 https://github.com/microsoft/vscode/issues/102052.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/102052#event-3570748127, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AO6SDYBRLHJCZ5ADBBTAHULR4V3DRANCNFSM4OWDZ4MA
.


168 emails

SEND ALL THE MAILS

Looking good! Waiting for @HanrichPotgieter to close :+1:

Great job. The emails are working and estimations look good.


Not enough info [TEST[

Type of issue
Missing information
Feedback
This page needs more info.
Page URL
https://learn.microsoft.com/en-us/windows/apps/develop/windows-integration/integrate-sharesheet-overview
Content source URL
https://github.com/MicrosoftDocs/windows-dev-docs/blob/docs/hub/apps/develop/windows-integration/integrate-sharesheet-overview.md
Author
@stevewhims
Document Id
f3ddfb51-bbed-2bed-96fe-e1e10a2e1190

What do you mean? What do you feel is missing?


jvmrun does not point to correct .jar file of the release

jvmrun in the release 1.0.3 points to the 1.0.0 jar file hence is not started by the ewon
jvmrun from release 1.0.3 zipfile:
-heapsize 25M -classpath /usr/flexy-cumulocity-connector-1.0.0-full.jar -emain com.hms_networks.sc.cumulocity.CConnectorMain # # #

Hi @ksbn7,
Thank you for the report. This bug has been resolved in the newly released version 1.1.0 of the connector.
-Alex


How scoped threads work: the role of the borrow checker

crossbeam-utils docs question
(Rust newbie asking.) Trying to better understand the difference between crossbeam and std threads I found the explanation
https://github.com/crossbeam-rs/crossbeam/blob/8cc906b9933ef443331c30513026346399debd2d/crossbeam-utils/src/thread.rs#L78-L79
in which I don't quite follow how that guarantee is enforced. I was under the impression that the interaction with the borrow checker happened in a conversion to the 'static lifetime of the closure the thread will execute
https://github.com/crossbeam-rs/crossbeam/blob/8cc906b9933ef443331c30513026346399debd2d/crossbeam-utils/src/thread.rs#L426-L427
but I don't understand (and possible I'm misinterpreting the documentation) if the borrow checker is actually enforcing the limited scope of the thread or if this is something just guaranteed by the API, meaning I can remove the logic that joins the threads at the end of the scope()
https://github.com/crossbeam-rs/crossbeam/blob/8cc906b9933ef443331c30513026346399debd2d/crossbeam-utils/src/thread.rs#L171-L176
and the program will still build successfully (the tests will fail of course, but the compiler, and particularly the borrow checker, won't complain).
(Just to clarify, I'm not saying the mechanism is incorrect or that it doesn't work, this is just a question from an inexperienced Rust, and Crossbeam, developer trying to better understand how the system works; if there is anything worth clarifying in the documentation I'll happily submit a PR about it).

but I don't understand (and possible I'm misinterpreting the documentation) if the borrow checker is actually enforcing the limited scope of the thread or if this is something just guaranteed by the API

Borrow checking is just a part of type checking. Lifetimes are actually types and therefore all the borrow checker does is it verifies that all the types/lifetimes match as expected.
Crossbeam's scoped threads are a new abstraction built with unsafe code, where it is allowed to transmute types and lifetimes into other arbitrary types and lifetimes. And as long as the public API is sound, everything is all right.

and the program will still build successfully (the tests will fail of course, but the compiler, and particularly the borrow checker, won't complain).

But then the scoped threads would be unsound, i.e. broken :) The idea is that we build an API for scoped threads that can't be misused from the outside.
The reason why you're able to delete this code and still make it compile is because there's other unsafe code within the implementation of scoped threads that makes this possible. But that doesn't mean deleting this code is a safe thing to do!

Hey @stjepang, thanks for the detailed answer. I understand and agree with everything you say, the point I was trying to make was targeted towards the documentation (a label I would add to this issue), where the phrase we guarantee to the borrow checker might give the impression that the borrow checker is actually the one verifying that the thread is indeed scoped, which from your previous comment I understand it doesn't.
It seems that that is the responsibility of the Crossbeam API, which is taking charge and "disabling" the lifetime check using a conversion to 'static and applying other checks itself. This is a subtle and unimportant distinction, I'm just mentioning it because it may help inexperienced Rust developers (as myself) to better distinguish the boundaries between what is the Rust compiler able to do and what is Crossbeam adding on top of it.


Doesn't Open at Startup

On Windows 10 it will not open on startup. Syncthing runs in the background but SyncTrayzor does not open, therefore no notifications.

Syncthing runs in the background but SyncTrayzor does not open

Wait, so syncthing.exe is running, but SyncTrayzor.exe isn't? Either SyncTrayzor is not set to start on login, but something else is starting Syncthing, or SyncTrayzor is starting Syncthing and then silently crashing.
What happens if you try and launch SyncTrayzor (e.g. from the start menu)?

In settings SyncTrayzor is set to automatically start on login. I have no syncthing service, it starts at login, not startup. Windows 10 doesn't have the same Startup Tab of msconfig as 7 but the task manager has SynTrayzor as Enabled Low.
Next time I reboot I'll take a look at the task manager when I start up.

Just try starting SyncTrayzor now: does it start successfully?

Yes, it opens fine.

Ok so something weird is going on. Can you open the log file and see if there are any entries at all from the time that you started your computer and logged on? You'll find the log file at C:\Users\<You>\AppData\Roaming\SyncTrayzor\Logs\SyncTrayzor.log. Paste the entire section of log file which covers those times in here. Thanks!

I finally got a chance to reboot.
Upon login the try icons is there and synctrayzor.exe and syncthing.exe are running (yay for hdd thrashing on startup!). After about 2-3 minutes the tray icon is no where to be found but both exe's are still running.
Reading through the log there was an update from v0.11.22 to v0.11.23 that was successful. Nothing else in the log except for syncing and scanning until I noticed the tray icon was missing.

OK, so everything's still running except the tray icon has disappeared. Are you sure Windows hasn't just tidied it away into the hidden tray icons seconds: try clicking the little 'up' arrow on the taskbar.
Could you post your log file, please?

Log: http://pastebin.com/2QMk9Vat

Sorry, this slipped off my radar.
Pastebin has deleted your log. Could you re-post, please?

Closing due to inactivity. Feel free to re-open.


How to reset the Kubernetes cluster with Torus

Hi, I am using Torus (v0.1.1) as a volume plugin in a Kubernetes (v1.2.3) Cluster with Postgres and Mongo volumes.
After creating my Pods, I get some errors like "status: Failure, reason: Unit torus-dev-nbd0.service already exists." etc. However, I am unable to somehow reset the cluster, to do the tests again. What do I have to do to reset the Torus setup (except recreating the whole cluster)?
(k8s-node.j2 is my Ansible node template, that I use to create the cluster)

I think we chatted some days ago on IRC, but just double checking how that worked.
Yeah, NBD is kind of a piece of work, and shortly you'll see an alternative for modern kernels (based on our new lib over at https://github.com/coreos/go-tcmu ). Killing the service with systemctl kill torus-dev-nbd0.service is sufficient, but, yeah, NBD disconnection, while it's gotten better, is still flaky with that kernel module sometimes.


Gaps in documentation (add getting started guide, update user guide and examples)

Some gaps within the documentation at the moment.

Need a Getting Started guide in order to facilitate different development scenarios (against bare-metal, minikube, minishift, etc.)
Update the User Guide with more information / elaborate on all commands
Highlight examples / make examples more prominent within Kedge.

PR for getting started guide added here #508

@kadel Creating something similar to https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands for examples + kedge CLI reference, what do you think?
We will essentially have two slate sections (similar to how they do it within http://kubernetes.io).
My thinking is that by creating a separate one for kedge CLI (converting http://kedgeproject.org/user-guide/ to slate) we can also elaborate / focus on examples.
WDYT?

LGTM

Going to close this :+1: Since I've completely each-task.


Refactor Odyssey to support WA state sites

A number of WA state counties use Odyssey.  The WA sites (we discovered on Snohomish) revealed a bug due to difference case metadata on search result pages, compared to metadata present on GA Dekalb/Chatham.
Need to refactor code to be more flexible for Odyssey search result pages.
Reproducing the bug
To reproduce the bug:
from court_scraper.platforms.odyssey_site import OdysseySite

url = "https://prdodypub.courts.wa.gov/ODYPORTAL/Home/Dashboard/29"
site = OdysseySite(url<EMAIL_ADDRESS>"PASSWORD", download_dir='/tmp')
site.login(headless=False)
results = site.search(search_terms=["19-2-11450-31"], headless=False)

Problem locale
# court_scraper/platforms/odyssey_site/pages/search_results.py
     64     def metadata(self):
     65         case_detail_url = self.el.find_element(
     66                 *SearchResultsPageLocators.CASE_DETAIL_LINK
     67             ).get_attribute('data-url')
     68         # FIX NEEDED HERE
     69         # Probably more flexible handling based on known column headers
     70         case_num, style_def, file_date, status, party_name = [
---> 71             field.strip() for field in self.inner_text.split('\t')
     72         ]
     73         return {

Stacktrace
Traceback (most recent call last):
  File "wa_portal.py", line 6, in <module>
    results = site.search(search_terms=["19-2-11450-31"], headless=False)
  File "/Users/tumgoren/code/stanford/biglocal/court-scraper/court_scraper/platforms/odyssey_site/site.py", line 44, in search
    row_data = case_row.metadata
  File "/Users/tumgoren/code/stanford/biglocal/court-scraper/court_scraper/platforms/odyssey_site/pages/search_results.py", line 69, in metadata
    field.strip() for field in self.inner_text.split('\t')
ValueError: not enough values to unpack (expected 5, got 4)

The fix for #69 required more flexible handling of header rows, which should in theory set us up to handle the variability in Washington. We'll need to ensure that the CaseInfo mappings (the raw field names to standardized field names) are handled for whatever is available in WA

@justinmayo Note that both of our accounts appear to have been locked by this portal, so I'm unable to test with your credentials or mine. Not sure if this is a recent development, but it appears the portal is CAPTCHA protected for unregistered users and requires you to contact court administrators to register for access to documents (and potentially other, more sensitive data?).


Speed to process 11MB of text into vector database

I am creating a vector database using this hardware:
Hardware:
    LENOVO_MT_82WK_BU_idea_FM_Legion Pro 5 16IRX8 32GB
    NVIDIA GeForce RTX 4070 8GB

Text data is as follows:
Data Files:
   Total # text files: 3,482 files
       Total filesize: 11,626,546 bytes
     Average filesize: 3,339. bytes
      Median filesize: 2,908 bytes
    Smallest filesize: 36 bytes
     Largest filesize: 245,026 bytes

I am using default models to process this data.
To process this vector database took ~32 hours.
Am I "holding it wrong"?

It took me many hours to process a csv of around 2mb... we are also experiencing this issue

It’s common for local models to process data slowly, especially when using smaller models.

我监听了上传的Tokens，每1500字符(在测试中主要是中文字符),使用了10758字符的提示词（1.2k单词）。返回的结构很完美。我想我可能找到速度缓慢缓慢的原因了。某种意义上这个速度可能是合理的╮(╯_╰)╭

English translation for jingerhuang above:
I also encountered slow speed issues. I used "locally implemented text vectorization functions" and "remote large model interface (deepseek)". Based on hardware usage analysis, the problem appears to be in the large model processing part, not the vectorization part. According to the remote interface statistics, each input is about 3k tokens, output is 1.2k tokens, and each processing takes 60-80 seconds, which is normal for large model processing speed. The issue seems to be why it needs to process so many tokens. A text document of 123,579 characters (357KB) generated a demand of (3+1.2)*89k tokens, which I don't think is reasonable. There might be something affecting the performance.
The key points are:

Using local vectorization and remote deepseek model
Input: ~3k tokens, Output: ~1.2k tokens per request
Processing time: 60-80 seconds per request
Document size: 123,579 characters (357KB)
Total tokens: (3+1.2)*89k tokens
The token count seems unusually high for the document size

Here's the English translation:
I monitored the uploaded tokens and found that for every 1,500 characters (mainly Chinese characters in the test), it used 10,758 characters of prompt text (1.2k words). The returned structure was perfect. I think I may have found the reason for the slow speed. In a way, this speed might actually be reasonable ╮(╯_╰)╭
Analysis:

Each chunk of 1,500 Chinese characters requires a large prompt (10,758 characters/1.2k words)
The prompt size is significantly larger than the input text
This explains the high token count and processing time
While slow, it's actually working as expected given the prompt size
The emoji ╮(╯_╰)╭ suggests a resigned acceptance of this reality

This suggests that the performance bottleneck is due to the necessary but large prompt size, rather than any technical issues or inefficiencies in the implementation.


Fix: Inconsistent behavior when multiple DNS servers provided to fanout

Signed-off-by: denis-tingajkin<EMAIL_ADDRESS>
Description

Motivation and Context
https://github.com/networkservicemesh/networkservicemesh/issues/1659
How Has This Been Tested?

[X] Covered by existing integration testing
[X] Added unit testing to cover
[X] Tested locally
[ ] Have not tested

Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@denis-tingajkin - do you have a pointer to an nsm-coredns image that I can try directly without rebuilding locally?

@john-a-joyce no, I don't have the image.
If you want to test then you need to rebuild the image locally based on these changes.

@denis-tingajkin - I tested this change in my environment by CHERRY-PICKING these two commits.  I want to emphasize the cherry pick as I was able to test an image build from your branch without corrupting my environment.   The behavior is better but I noticed a slightly different issue.   I will append the details to the issue

@john-a-joyce thanks for testing. It can be fixed with the last changes. Please check if you have a chance.


Teleport Data

Data on all the different teleport types recorded for RS3
https://docs.google.com/spreadsheets/d/16h4qJwfHFd7aBtrE4hEqRl1w23WOlQgsnijx5-_lBgs/edit#gid=0

#67

#67


Documentation updates for 11.0.0 release

Which issue does this PR close?

N/A
Rationale for this change

Making some small improvements before we release
What changes are included in this PR?

Combine the two CLI pages into a single page and improve formatting
Move expressions out of DataFrame page and into a new page
Remove Ballista content
Fix link to sqlparser source
Rename "Community" section to  "Developer Resources" and moved link to docs.rs there
Fix broken link to communication page
Renamed one of the roadmap pages so we don't have two "roadmap" links

This is how the index looks now:

Are there any user-facing changes?

No

@kmitchener I'm planning on cutting the 11.0.0 release candidate in the next few days, so went ahead and made some small docs improvements here. Apologies if this overlaps a bit with some of your efforts.

Codecov Report

Merging #3129 (fd54cd7) into master (b1765f7) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3129      +/-   ##
==========================================
- Coverage   85.95%   85.94%   -0.01%
==========================================
  Files         291      291
  Lines       52382    52382
==========================================
- Hits        45025    45022       -3
- Misses       7357     7360       +3

Impacted Files
Coverage Δ

datafusion/core/src/physical_plan/metrics/value.rs
86.93% <0.00%> (-0.51%)
:arrow_down:

datafusion/expr/src/logical_plan/plan.rs
77.43% <0.00%> (-0.35%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Converted to draft because this is in conflict with https://github.com/apache/arrow-datafusion/pull/3128

I am going to break this down into smaller PRs.

@kmitchener I'm planning on cutting the 11.0.0 release candidate in the next few days, so went ahead and made some small docs improvements here. Apologies if this overlaps a bit with some of your efforts.

ok, I'll hold off on anything more until this latest flurry of changes gets committed. it's looking good!


Add a license to the repository

I recently came across your project BB84-Protocol-for-QKD and I noticed that the repository github.com/strangequarkkk/BB84-Protocol-for-QKD currently does not have a license specified.
Why Adding a License is Important?
As GitHub documents

Public repositories on GitHub are often used to share open source software. For your repository to truly be open source, you'll need to license it so that others are free to use, change, and distribute the software.

Would be possible to add a valid license to this repository? Probably something really OpenSource such as the Apache 2.0 license would fit really well.
Thanks!

Added the Apache 2.0 license.

thanks


iPad: don't flash blue background while rotating

@keybase/react-hackers
With rotation on, the new screen area being created during an iPad rotation is first filled in blue by iOS, and then rendered by JS, so there's a momentary observable flash of blue.
Setting the rootView's background color to transparent seems to work fine as a fix -- dark mode and light mode both have no visible glitch on rotation.  Initial app startup and splash screen look good to me too.  But maybe there's a reason I'm missing for why we want a colored bg?

mmmm, maybe look at the blame, its been awhile since i thought about this.
Things to check:

The 'hiding' action works when backgrounding correctly
No additional overdraw

@chrisnojima Checked those two, I think we're good with this change.  The choice of blue dates back to an early dark mode PR (#19987) and was to avoid flashing against the keys background image.  I don't see any flashing with this PR, I can keep an eye out after it goes into master too.


data object has to be duplicated in remote message

When using the notification service extension helper,  the data property has to be included in notifee_options. This can cause unnecessary duplication in the payload and an increase in payload size.

do we still have to include data in notifee_options? If I do, I get the error "The resulting APNs payload size is larger than the 4KB limit of APNs." Probably because I am repeating the data, @helenaford do you have any suggestions?


Issue templates

This (meta-)commit makes it possible for users of Compiler Explorer to indicate if their issue is a feature request or a bug (or something else).

I would love it if the issue template would contain something about making a shared link and pasting that, because that's usually more useful than a screenshot or reproduce step description

I think you can leave empty, as we already get notified in real time with any new issues :)
Thanks for taking the time to improve the user experience of the CE users :)

I'll merge this now. Sorry that it took so long to get addressed!


check-links is missing redirects

$ curl -D - https://rexray.thecodeteam.com/
HTTP/2 301
date: Thu, 19 Apr 2018 16:05:12 GMT
cache-control: max-age=3600
expires: Thu, 19 Apr 2018 17:05:12 GMT
location: https://rexray.io
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 40e09c80ac879278-EWR

But it was not being flagged by check-links.

Closed with https://github.com/cncf/landscape/commit/e8fc8dc146d99ca2f8005dc484f1f4a1cc88a4b0


[SPARK-27401][SQL] Refactoring conversion of Timestamp to/from java.sql.Timestamp

What changes were proposed in this pull request?
In the PR, I propose simpler implementation of toJavaTimestamp()/fromJavaTimestamp() by reusing existing functions of DateTimeUtils. This will allow to:

Simply implementation of toJavaTimestamp(), and handle properly negative inputs.
Detect Long overflow in conversion of milliseconds (java.sql.Timestamp) to microseconds (Catalyst's Timestamp).

How was this patch tested?
By existing test suites DateTimeUtilsSuite, DateFunctionsSuite, DateExpressionsSuite and CastSuite. And by new benchmark for export/import timestamps added to DateTimeBenchmark:
Before:
To/from java.sql.Timestamp:               Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
From java.sql.Timestamp                             290            335          49         17.2          58.0       1.0X
Collect longs                                      1234           1681         487          4.1         246.8       0.2X
Collect timestamps                                 1718           1755          63          2.9         343.7       0.2X

After:
To/from java.sql.Timestamp:               Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
From java.sql.Timestamp                             309            316           8         16.2          61.9       1.0X
Collect longs                                      1410           2747        1158          3.5         282.0       0.2X
Collect timestamps                                 1727           1818          96          2.9         345.3       0.2X

Can one of the admins verify this patch?

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9676/
Test PASSed.

Test build #104344 has started for PR 24311 at commit b0ced9f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9677/
Test PASSed.

Test build #104345 has started for PR 24311 at commit 6129c9a.

Test build #104344 has finished for PR 24311 at commit b0ced9f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104344/
Test FAILed.

Test build #104345 has finished for PR 24311 at commit 6129c9a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104345/
Test FAILed.

Failed ORC related test which generates random dates - HiveOrcHadoopFsRelationSuite. It seems ORC uses its own methods for converting Catalyst's days since epoch to something like java.sql.Date that is based on Julian calendar for old dates:
!== Correct Answer - 10 ==    == Spark Answer - 10 ==
 struct<index:int,col:date>   struct<index:int,col:date>
 [1,3477-08-12]               [1,3477-08-12]
 [2,7867-09-20]               [2,7867-09-20]
 [3,null]                     [3,null]
 [4,6577-03-26]               [4,6577-03-26]
 [5,8002-07-25]               [5,8002-07-25]
 [6,2154-05-10]               [6,2154-05-10]
 [7,4921-05-03]               [7,4921-05-03]
![8,0647-07-01]               [8,0647-06-28]
 [9,9422-12-25]               [9,9422-12-25]
 [10,2677-06-15]              [10,2677-06-15]

@dongjoon-hyun Am I right? It is strange that only this ORC test failed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9679/
Test PASSed.

Test build #104347 has started for PR 24311 at commit bf4b6b6.

Thank you for pinging me, @MaxGekk . I'll take a look today.
BTW, @srowen . Sorry, but do you mind if I ask why you approve this PR suffering from the UT failures?

Oh, really I mean this looks fine pending tests. If they're transient failures, fine. If they're not, obviously they need to be fixed. here I assume they aren't significant failures, just tests that need to be adjusted.

Oh, @MaxGekk . According to the last reverting commit, you are not aiming DATE in this PR. Are you going to file a JIRA issue for DATE type?

Thank you, @srowen . I see~
It seems that @MaxGekk reduced the scope of this PR first to pass the Jenkins.

Before the last reverting, I got the following result. @MaxGekk . If there was an issue, it might be an old Hive 1.2.1 issue which is fixed in Apache ORC. If needed, you can file a JIRA issue. For old Hive 1.2.1 issue, I can dig it for you but it might be difficult to fix it because it seems to be not inside both Apache Spark and Apache ORC.

OrcHadoopFsRelationSuite passed.
HiveOrcHadoopFsRelationSuite failed.

Test build #104347 has finished for PR 24311 at commit bf4b6b6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104347/
Test PASSed.

it because it seems to be not inside both Apache Spark and Apache ORC.

OrcHadoopFsRelationSuite passed.
HiveOrcHadoopFsRelationSuite failed.

Yeh, it seems there is a calendar incompatibility issue somewhere inside Hive + ORC which could cause the difference ![8,0647-07-01]               [8,0647-06-28] for old days before 1582 (I guess). I just thought we switched to Proleptic Gregorian calendar everywhere in Spark (https://issues.apache.org/jira/browse/SPARK-26651).
In any case, I think my date (not timestamp) related changes are potentially more expensive comparing to current implementation because of conversion of java.sql.Date -> java.time.LocalDate. The conversion extracts components like year, month and etc from java.sql.Date that are not cheap but current implementation does time zone shifting which is not necessary too. In any case, I am going to leave Java <-> Catalyst's date conversions as is for now.

Test build #104360 has started for PR 24311 at commit 20dfacf.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9692/
Test PASSed.

Test build #104360 has finished for PR 24311 at commit 20dfacf.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104360/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9697/
Test PASSed.

Test build #104365 has started for PR 24311 at commit ec15d82.

Test build #104365 has finished for PR 24311 at commit ec15d82.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104365/
Test FAILed.

Merged build finished. Test FAILed.

Retest this please.

Test build #104399 has started for PR 24311 at commit ec15d82.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9731/
Test PASSed.

Test build #104399 has finished for PR 24311 at commit ec15d82.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104399/
Test PASSed.

Test build #104421 has started for PR 24311 at commit b4cade3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9752/
Test PASSed.

Test build #104421 has finished for PR 24311 at commit b4cade3.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104421/
Test FAILed.

jenkins, retest this, please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9756/
Test PASSed.

Test build #104425 has started for PR 24311 at commit b4cade3.

Test build #104425 has finished for PR 24311 at commit b4cade3.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104425/
Test FAILed.

jenkins, retest this, please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9762/
Test PASSed.

Test build #104431 has started for PR 24311 at commit b4cade3.

Test build #104431 has finished for PR 24311 at commit b4cade3.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104431/
Test FAILed.

I changed only sql/core/benchmarks/DateTimeBenchmark-results.txt since the last success run https://github.com/apache/spark/pull/24311#issuecomment-481012862 . @dongjoon-hyun @srowen Does it make sense to re-run this again? It seems there is a permanent problem with python tests.

jenkins, retest this, please

Test build #104449 has started for PR 24311 at commit b4cade3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9779/
Test PASSed.

Test build #104449 has finished for PR 24311 at commit b4cade3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Finally, it passed! Merged to master. Thank you, @MaxGekk and @srowen !

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104449/
Test PASSed.


[Snyk] Upgrade @typescript-eslint/eslint-plugin from 6.20.0 to 6.21.0

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade @typescript-eslint/eslint-plugin from 6.20.0 to 6.21.0.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 21 versions ahead of your current version.
The recommended version was released 21 days ago, on 2024-02-05.

Release notes

Package name: @typescript-eslint/eslint-plugin

6.21.0 - 2024-02-056.21.0 (2024-02-05)
🚀 Features

export plugin metadata (#8331)
allow parserOptions.project: false (#8339)
eslint-plugin: add rule prefer-find (#8216)
typescript-estree: forbid duplicated accessibility modifiers (#8257)
utils: improve eslint types (#8344)

🩹 Fixes

ast-spec: add JSXElement type to the JSXAttribute['value'] (#8285)
eslint-plugin: [no-unused-vars] don't report on types referenced in export assignment expression (#8265)
eslint-plugin: [switch-exhaustiveness-check] better support for intersections, infinite types, non-union values (#8250)
eslint-plugin: [consistent-type-imports] dont report on types used in export assignment expressions (#8332)
eslint-plugin: [no-unnecessary-condition] handle left-hand optional with exactOptionalPropertyTypes option (#8249)
eslint-plugin: [class-literal-property-style] allow getter when same key setter exists (#8277)
eslint-plugin: [no-unnecessary-type-assertion] provide valid fixes for assertions with extra tokens before as keyword (#8326)
rule-tester: fix a phantom dependency on the "semver" package (#8260)

❤  Thank You

auvred @ auvred
Brad Zacher @ bradzacher
Kirk Waiblinger @ kirkwaiblinger
Pete Gonzalez @ octogonz
YeonJuan @ yeonjuan

You can read about our versioning strategy and releases on our website.

6.20.1-alpha.19 - 2024-02-05

6.20.1-alpha.18 - 2024-02-04

6.20.1-alpha.17 - 2024-02-04

6.20.1-alpha.16 - 2024-02-04

6.20.1-alpha.15 - 2024-02-04

6.20.1-alpha.14 - 2024-02-03

6.20.1-alpha.13 - 2024-02-03

6.20.1-alpha.12 - 2024-02-03

6.20.1-alpha.11 - 2024-02-03

6.20.1-alpha.10 - 2024-02-03

6.20.1-alpha.9 - 2024-02-03

6.20.1-alpha.8 - 2024-02-03

6.20.1-alpha.7 - 2024-02-02

6.20.1-alpha.6 - 2024-02-02

6.20.1-alpha.5 - 2024-02-02

6.20.1-alpha.4 - 2024-02-01

6.20.1-alpha.3 - 2024-02-01

6.20.1-alpha.2 - 2024-01-30

6.20.1-alpha.1 - 2024-01-30

6.20.1-alpha.0 - 2024-01-29

6.20.0 - 2024-01-296.20.0 (2024-01-29)
🚀 Features

eslint-plugin: [member-ordering] allow easy reuse of the default ordering (#8248)

🩹 Fixes

eslint-plugin: [no-useless-template-literals] incorrect bigint autofix result (#8283)
eslint-plugin: [prefer-nullish-coalescing] treat any/unknown as non-nullable (#8262)
eslint-plugin: [no-useless-template-literals] report Infinity & NaN (#8295)
eslint-plugin: [prefer-readonly] disable checking accessors (#8300)

❤  Thank You

Alex Parloti @ parloti
auvred @ auvred
James Browning
StyleShit @ StyleShit
YeonJuan @ yeonjuan

You can read about our versioning strategy and releases on our website.

    from @typescript-eslint/eslint-plugin GitHub release notes

Commit messages

Package name: @typescript-eslint/eslint-plugin

289ee88 chore(release): publish 6.21.0
847d193 docs: add FAQ entry about ESLint's --cache (#8372)
f892a72 docs(eslint-plugin): mention alternatives in no-explicit-any (#8370)
aa5edf7 chore(deps): update dependency prettier to v3.2.4 (#8357)
73669b6 chore(deps): update dependency markdownlint-cli to ^0.39.0 (#8354)
0153379 chore(deps): update dependency @ types/jest to v29.5.12 (#8371)
9cb8caf chore(deps): update dependency @ babel/eslint-parser to v7.23.10 (#8349)
664a3d4 chore(deps): update dependency webpack to v5.90.0 (#8359)
f571aec chore(deps): update dependency netlify to v13.1.14 (#8353)
204cfac chore(deps): update dependency @ types/react to v18.2.51 (#8361)
33185ca chore(deps): update dependency @ types/node to v20.11.15 (#8356)
af7183c chore(deps): update dependency ignore to v5.3.1 (#8360)
da26790 chore(deps): update dependency @ swc/jest to v0.2.34 (#8363)
a893105 feat: allow `parserOptions.project: false` (#8339)
47d25f4 fix(rule-tester): fix a phantom dependency on the "semver" package  (#8260)
1d7447f chore(eslint-plugin): [no-invalid-void-type] fix `Options` typing to reflect `minItems: 1` (#8312)
e2b46e0 fix(eslint-plugin): [no-unnecessary-type-assertion] provide valid fixes for assertions with extra tokens before `as` keyword (#8326)
2373a63 fix(ast-spec): add `JSXElement` type to the `JSXAttribute['value']` (#8285)
2f99e70 chore(deps): update dependency lint-staged to v15.2.1 (#8350)
68e1e8b chore(deps): update dependency react-resizable-panels to ^0.0.63 (#8328)
e30e76b chore(deps): update dependency monaco-editor to ~0.45.0 (#8161)
767d689 chore(deps): update dependency markdownlint-cli to ^0.38.0 (#8159)
c097bfc chore(deps): update dependency prettier to v3.2.4 (#8127)
ba2f166 chore(deps): update dependency lint-staged to v15.2.0 (#8043)

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

fixed in #187


Update referential integrity and referential actions documentation

Describe this PR
I've made initial changes to the referential integrity and referential actions docs based on conversations with @tanberry, @pimeys and @dpetrick yesterday. At the moment this is mainly getting up-to-date information in there - we still need to make the organisation of the pages as a whole better.
Comments welcome!
Changes

explained difference between emulating in client and handling in the database
added info on MongoDB support
some copyediting/formatting changes

What issue does this fix?
WIP for issue #2538
Any other relevant information

Now live at https://www.prisma.io/docs/concepts/components/prisma-schema/relations/referential-integrity + https://www.prisma.io/docs/concepts/components/prisma-schema/relations/referential-actions


Modular usage

Would be great if he methods could be imported separately to decrease overall app size

I agree, the current size is a little bit big.

yeah looking forward to this.

Just made lean-he for my use case, forked from this very repo. Please see if this can help others.

@adnaan1703 how big is lean-he?

also if you need only decode method you can use entity-decode, it is only 373B for browser, but if it is used in the node.js it loads he and for node.js size doesn't matter

@mdevils' html-entities library is a little smaller, about as popular in terms of NPM weekly downloads, and more recently updated.


Question: How certificate rotation works with fleet provisioning feature using aws iot device client?

Description: I am using aws-iot-device-client for a couple devices with fleet provisioning feature. When using fleet provisioning, a runtime configuration is created when thing is successfully registered in AWS IOT Core like below.
    "completed-fp": true,
    "cert": "/path/to/newly/created/certificate.pem.crt",
    "key": "/path/to/private.pem.key",
    "thing-name": "NewlyProvisionedThingName"
    }
}

If I would like to rotate certificates(x.509, private key), I am either deleting the above configuration file or changing the "completed-fp" parameter to false. Per documentation when the "completed-fp" is false, aws-iot-device-client will re-provision the device which in turn generates a new set of certificates for the device. But the problem is  old certificates will still live on the device and are not removed from AWS IOT Core.
Question: While using aws-iot-device-client with fleet provisioning feature, Could you please provide me the best way on how to rotate certificates and automate the process. Also, Is re-provisioning the device will cause any issues and is it fine to do it?
Initial thoughts: My initial thought is to create a lambda function and follow below steps

Create AWS IOT Job to change/remove runtime configuration file and restart aws-iot-device-client service.
Create AWS IOT Job to delete the old certificates by taking certificate name and location as input parameter on the device.
Lambda function to receive input on which devices to rotate certificates and run below jobs.
Run the above created job on the each device specified which will generate a new set of certificates by reprovisioning the device.
Once job is successful, Query for the list of certificates and remove everything based on date created except the new one and run another job to delete the old certificates
Finally remove the certificates on the IOT Core based on date created.

This process looks a bit sketchy, so wanted to know what will be best way to rotate certificates.
Thank you!

Hi @Kammula280,
Thank you for reaching out. Automated certificate rotation is something we want to support in the future, but we don't currently have a timeline!
I think your idea outlined above would work.
However, re-provisioning the device will result it in provisioning a whole new IoT Thing. So, it is more like starting over than rotating the certificate.
The way to actually rotate the certificates is outlined here: https://aws.amazon.com/blogs/iot/how-to-manage-iot-device-certificate-rotation-using-aws-iot/
This would require some development on the Device Client side however. This is on my radar now but unfortunately I can't make any guarantees as to when it might be available.
You can of course attempt to do a custom implementation of this building on top of the Device Client and are welcome to ask any questions here and the team will answer.
On the cloud side you would need to setup Device Defender, SNS, and Lambda like it outlines in the above link.
On the Device Side:

Need to subscribe to management/topic/{clientId}/csr_req and add code for generating a Certificate Signing Request and publishing it to management/topic/{clientId}/csr_res

Need to subscribe to management/topic/{clientId}/crt and add code for attempting to re-establish MQTT connection when new certificate is received on that topic. If successful, update config to point to new certificate, delete old certificate, publish to management/topic/{clientId}/crt_ack. If not successful, publish to management/topic/{clientId}/crt_err with error.

You could try doing this with the Pub/Sub sample feature gives examples of how to Pub/Sub to topics.
Hope that helps!

Hi @shangabl
Thank you for your quick response and I really appreciate your effort on iot-device-client. Glad now that you have certificate rotation on your radar now. I am not familiar with cpp to perform the Device Side changes. I have implemented the below approach for device certificate rotation and will put some notes here if it helps anyone.
During the implementation, I found a couple issues. If you believe they are bugs and need a separate issue, let me know I will open separate issues.
Issue1:
While using fleet provisioning feature, Once the thing is registered a runtime config file is generated(~/.aws-iot-device-client/aws-iot-device-client-runtime.conf). This file consists of an additional character in the end which causes issues while parsing json.
Example file with additional charcter in the last line
{
"runtime-config": {
    "completed-fp": true,
    "cert": "/root/.aws-iot-device-client/keys/61ab473dd81f24753c5d866d7babe1f4b47dd293d3232-certificate.pem.crt",
    "key": "/root/.aws-iot-device-client/keys/fbrhvnd866d7babe1f4b47dd293d3232-private.pem.key",
    "thing-name": "final_final13",
    "device-config": {

        }
    }
}**^@**

Issue2:
While trying to rotate certificates in the device, I am getting new certificates calling AWS API and changing the runtime configuration file by replacing the  "cert" and "key" with new certificate path.
Goal Here is to reload the aws-iot-device-client with new configuration file.
To achieve reloading of aws-iot-device-client with new configuration file, I am trying to restart the aws-iot-device-client using jobs feature. Issue here is when try to run the job to restart the aws-iot-device-client, the job never completes and the aws-iot-device-client restart is ending in an infinite loop.
Approach I tried to restart aws-iot-device-client using jobs feature is

Create another main process/subprocess within the job handler and finish the job process but the job process is not quitting till main process/subprocess are finished.
From job handler, create a shell script file and set permissions. Execute the shell script as sub process or another main process

#!/usr/bin/env sh
set -e
sleep 30
sudo systemctl restart aws-iot-device-client

The job is never completed and ends up in infinite loop.

Question How can achieve the reload the aws-iot-device-client with new configuration file? Also, Do you think there will be a need for the restart of aws-iot-device-client from jobs feature if the user wants to? I am not sure if there is any heart beat check and aws-iot-device-client can restart itself if connection to aws fails.
I can provide some log files and any other information needed if this is not expected behavior.
Implementation of Device certificate rotation notes

On Cloud side, setup Device defender and SNS alerts of Device certificate expiration check to be sent to Lambda function.
Some prerequisites will be setting up iot role alias so device can get aws credentials to call apis.
Lambda function to execute a job on the device.
Job document

{
  "version": "1.0",
  "steps": [
    {
      "action": {
        "name": "test",
        "type": "runHandler",
        "input": {
          "handler": "rotatecert.py",
          "args": [
            " policy name",
            "Credentials provider url",
            "region"
          ],
          "path": "/etc/.aws-iot-device-client/jobs/"
        },
        "runAsUser": "ubuntu"
      }
    }
  ]
}

Job Handler, Does the following
a. Read the runtime configuration file
b. Get AWS credentials using iot credentials endpoint
c. Generate new certificates
d. Save the certificate files and set permissions
e. replace the runtime configuration file with new certificate paths.
f. Reload the aws-iot-device-client to pickup new configuration (Not able to achieve this)
g. Revoke the old certificates

#!/usr/bin/env python3
import sys
import boto3
import urllib3
import json
import subprocess
import os

policy_to_attach=sys.argv[2]
credentials_url = sys.argv[3]
region = sys.argv[4]

try:
    device_iot_config_file = open('/root/.aws-iot-device-client/aws-iot-device-client-runtime.conf')
    device_iot_config_data = json.load(device_iot_config_file)
    device_iot_config_file.close()
except:
    device_iot_config_file = open('/root/.aws-iot-device-client/aws-iot-device-client-runtime.conf')
    data = device_iot_config_file.read()
    new=data[:-1]
    device_iot_config_data= json.loads(new)
    device_iot_config_file.close()

cert_file =device_iot_config_data["runtime-config"]["cert"]
key_file = device_iot_config_data["runtime-config"]["key"]
thing_name = device_iot_config_data["runtime-config"]["thing-name"]
keys_folder='/root/.aws-iot-device-client/keys/'
iot_client_restart_script_folder='/etc/.aws-iot-device-client/jobs/'

retries = urllib3.Retry(connect=5, read=2, redirect=5)
http = urllib3.PoolManager(
    cert_file=cert_file,
    key_file=key_file,
    cert_reqs="CERT_REQUIRED",
    headers={
        'x-amzn-iot-thingname': thing_name
    },
    retries=retries
)

def get_boto_session():
    print(credentials_url)
    resp = http.request("GET", credentials_url)
    print(resp.status)
    if resp.status != 200:
        raise Exception("Failed to retrieve STS token: {}".format(resp.reason))
    creds = json.loads(resp.data)['credentials']
    return boto3.session.Session(aws_access_key_id=creds['accessKeyId'], aws_secret_access_key=creds['secretAccessKey'], aws_session_token=creds['sessionToken'], region_name=region)

def list_thing_principals(thingName):
    response = iot.list_thing_principals(
    thingName=thingName
    )
    return response["principals"]

def create_keys_and_certificate(setAsActive):
    response = iot.create_keys_and_certificate(
    setAsActive=setAsActive
    )
    return response

def create_keys_and_certificate_files(filename, content):
    writetofile = open(filename, "w")
    writetofile.write(content)
    writetofile.close()
    return "success"
def create_keys_and_certificate_files_permissions(path, permission):
    os.chmod(path, permission)
    return "success"

def attach_policy(policyName, target):
    response = iot.attach_policy(
        policyName=policyName,
        target=target
    )
    return "success"

def attach_thing_principal(thingName, principal):
    response = iot.attach_thing_principal(
    thingName=thingName,
    principal=principal
    )
    return "success"

def replacetext(filename, old_text,new_text):
    writetofile = open(filename, "r")
    data = writetofile.read()
    data = data.replace(old_text, new_text)
    writetofile.close()

    writetofile = open(filename, "w")
    writetofile.write(data)
    writetofile.close()
    return "success"

#syncronous operation will wait for child process to finish, for asyncronous use Popen
# def restart_service():
#     commands = ["sudo", "systemctl", "restart", "aws-iot-device-client.service"]
#     subprocess.run(commands, shell=False, check=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
#     return "success"

def Execute_shell_script(shell_script_path):
    subprocess.Popen(["sh", shell_script_path],shell=False,stdin=None, stdout=None, stderr=None, close_fds=True)
    return "success"

def revoke_certificates(thing_name, new_certificate_certificateArn):

    #get list of certs for this thing
    certsResponse = iot.list_thing_principals(
        thingName = thing_name
    )

    #revoke and delete all certs associated with this thing except for the new one
    for certARN in certsResponse["principals"]:
        if certARN != new_certificate_certificateArn:
            #get cert id
            base,certificateId = certARN.split("/")

            #check its status
            statusResponse = iot.describe_certificate(
                certificateId = certificateId
            )

            if statusResponse["certificateDescription"]["status"] != "REVOKED":
                #revoke the certificate
                updateCertResponse = iot.update_certificate(
                    certificateId = certificateId,
                    newStatus = 'REVOKED'
                )
                print("Revoked cert: " + certificateId)

                #list the policies for the cert so they can be detached
                policiesResponse = iot.list_attached_policies(
                    target=certARN,
                    recursive=False
                )

                #remove the attachement
                for policy in policiesResponse["policies"]:
                    response = iot.detach_principal_policy(
                        policyName=policy["policyName"],
                        principal=certARN
                    )
                    print("Detached policy: " + policy["policyName"])

                #detach the cert from the thing
                response = iot.detach_thing_principal(
                    thingName=thing_name,
                    principal=certARN
                )

                #delete the certificate
                response = iot.delete_certificate(
                    certificateId=certificateId,
                    forceDelete=False
                )
                print("Delete cert: " + certificateId)

    return 'Done'

if __name__ == '__main__':
    sess = get_boto_session()
    iot = sess.client('iot')
    old_certificate_certificateId,base=cert_file.split('/')[-1].split('-')
    #base,old_certificate_certificateId=list_thing_principals(thing_name)[0].split('/')
    new_certificate_details =create_keys_and_certificate(True)
    new_certificate_certificateId= new_certificate_details["certificateId"]
    new_certificate_certificateArn= new_certificate_details["certificateArn"]
    new_certificate_certificatePem =new_certificate_details["certificatePem"]
    new_certificate_PrivateKey=new_certificate_details["keyPair"]["PrivateKey"]
    new_certificate_certificatePem_file=create_keys_and_certificate_files(keys_folder+new_certificate_certificateId+'-certificate.pem.crt', new_certificate_certificatePem)
    new_certificate_PrivateKey_file=create_keys_and_certificate_files(keys_folder+new_certificate_certificateId+'-private.pem.key', new_certificate_PrivateKey)
    new_certificate_certificatePem_file_permission=create_keys_and_certificate_files_permissions(keys_folder+new_certificate_certificateId+'-certificate.pem.crt', 0o644)
    new_certificate_PrivateKey_file_permission=create_keys_and_certificate_files_permissions(keys_folder+new_certificate_certificateId+'-private.pem.key', 0o600)
    attach_policy(policy_to_attach, new_certificate_certificateArn)
    attach_thing_principal(thing_name , new_certificate_certificateArn)
    replacetext("/root/.aws-iot-device-client/aws-iot-device-client-runtime.conf", old_certificate_certificateId,new_certificate_certificateId)
    #restart_service()
    iot_client_restart_script = '#!/usr/bin/env sh\nsleep 30\nset -e\nsudo systemctl restart aws-iot-device-client'
    iot_client_restart_script_file=create_keys_and_certificate_files(iot_client_restart_script_folder+'iot_client_restart_script.sh', iot_client_restart_script)
    iot_client_restart_script_file_permission=create_keys_and_certificate_files_permissions(iot_client_restart_script_folder+'iot_client_restart_script.sh', 0o700)
    #Execute_shell_script(iot_client_restart_script_folder+'iot_client_restart_script.sh')
    revoke_certificates(thing_name, new_certificate_certificateArn)

Thank you!

Hi @Kammula280,
Thanks for these notes!

I would say this is a bug. The additional character of the runtime configuration file is not something I have seen before. I'll attempt to recreate that issue and see why that is happening. We definitely do not want the runtime configuration file to be invalid JSON. What operating system / architecture are you using? I'll try to recreate using the same.
The issue with the infinite loop is due to Device Client restarting during execution of the Job. Device Client Gets Job -> Executes Job via Job handler -> Updates Job. But if it restarts it never updates the Job and then the Job is still pending and it will start over again. We have logic in the reboot.sh handler that deals with this situation. That is a little different because we need to get last boot time, uptime and figure out if we rebooted or not... I think here in your handler you could just use a lock file to track whether or not you have restarted. Check if file exists, if it does not then you restart, if it does exist then don't restart and delete the lock file.

Hi @shangabl
For Issue 1, additional character in runtime configuration file, Environment I have used is x86_64 GNU/Linux
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.6 LTS"
Just now, I also tested in below environment and the behavior is same.
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS"
AMI in AWS =ami-0a6b2839d44d781b2 ( x86_64)
For Issue2, I will try using a lock file an see if I am able to restart the aws-iot-device-client if it is only way to reload with a new runtime configuration file.
Thank you!

I attempted to recreate the issue and I just get valid JSON...
Are you saying it generates the Runtime config with the extra character and then fails to parse the runtime config file and crashes? Can you show error output? Thanks!

Hi @shangabl
When I use fleet provisioning feature, a runtime configuration file is created and iot-device-client is running successfully without any errors.
I am only seeing an additional character in the end of the runtime configuration file when use vi editor or when I download the file or when I am programmatically reading that file.

Hey @Kammula280 , thanks for posting those screenshots. I see the exact same thing when testing fleet provisioning from my side. "^@" is simply the manifestation of the null terminating character in the file. Small bug that we will push a quick fix for.

Hi @Kammula280,
I also merged a fix for the infinite loop of restarting Device Client service.
I will close this issue out now as we have merged fixes for both the issues you discovered.
Thank you for your help here!
Shane

Thank you very much Shane!

Hi @Kammula280,
I am re-opening this because I realized we had to revert the fix for the runtime config file fix. We should be merging the fix again shortly but we reverted due to a test failure!
This should be resolved shortly but I wanted to make sure you had the correct information!
Shane


[Bug Report] Upload Component cannot trigger file selector when using Chrome

Element UI version
2.15.6
OS/Browsers version
Chrome Version 94.0.4606.81 (Official Build) (arm64) / MacOS Big Sur 11.6 (20G165)
Vue version
2.6.11
Reproduction Link
https://jsfiddle.net/mmx38qxw
Steps to reproduce
just try it on the official web with Chrome Version 94.0.4606.81 (Official Build) (arm64) on MacOS Big Sur 11.6 (20G165) with Apple M1 chip
https://element.eleme.io/#/en-US/component/upload
What is Expected?
open the file selector
What is actually happening?
No response

https://github.com/DouyinFE/semi-design/issues/600 ， 解决方式: 升级 Chromium 内核到98


JAMES-3693 Rate limiting should allow choosing precision

Can we please apply precision in all test classes implying mailets and rate limiting... Both unit and integration tests.
We would benefit from the additional stability...

Can someone rerun the CI?


compatilibity with Kifu for Android

https://github.com/sunfish-shogi/electron-shogi/issues/265

Codecov Report
Base: 70.26% // Head: 70.26% // No change to project coverage :thumbsup:

Coverage data is based on head (fe4663c) compared to base (437c666).
Patch coverage: 50.00% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #275   +/-   ##
=======================================
  Coverage   70.26%   70.26%
=======================================
  Files          60       60
  Lines       11589    11589
  Branches     1565     1565
=======================================
  Hits         8143     8143
  Misses       1902     1902
  Partials     1544     1544

Impacted Files
Coverage Δ

src/shogi/kakinoki.ts
79.63% <50.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Updating adviser funnel start page

Trello card
https://trello.com/c/cM2bchoj/3478-update-the-tta-content-to-cover-the-revamped-eea-service
Context
Changes proposed in this pull request
Guidance to review

@gemmadallmandfe have made some changes based on feedback from Kat/Jade/Steph Roberts

@gemmadallmandfe these changes still need to go live! 😬


interop between WASM containers

this is the new docker in a way.
Is there an common interop between WASM containers yet ? Wondering if GRPC is the right way to go ?
I have not even tried but raising here as i expect the team there have thought about this.
There seems to be two fundamental approaches that i see so far:
1- Open a port on the VM host that lets the VM guest use anything it wants.
2- Integrate some sort of IO API at the host level that allows VMS to talk to each other using a formalised transport.
curious where you are heading on this.

It should be easy to implement IPC by exposing an API from the host that copies memory between different VMs.

sure but that seems foolhardy.
Basically i don't want to build my own because whats going on the under the hood is complex and i think its better is there is an "official" way.

I think using NATS is a decent choice to do IPC and node ot node.
what do you think ?


[WIP] models cleanup

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: pimeys:x: Marcus BöhmMarcus Böhm seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Automatically run benchmarks and fill README.md

[X] Run debug versions
[X] Fill start time / memory consumption in README
[x] Check results liability compared to manual benchmark
[x] Auto build app
[x] Fill build size in README
[x] Fill build time in README (#36)
[x] Run release versions
[x] Invalid macos build size

Some builds are missing, but I'll fill them in other PR, while updating examples


added workflow as per issue #19

Fixes #19
@BamaCharanChhandogi please review and merge this PR
Also, please add labels and assign me this PR
Thank You!

@BamaCharanChhandogi Done with all the changes

We don't want this feature. so I will think and I will merge.
Thanks.


REX-Ray Docker Plug-in for DigitalOcean Block Storage

This patch provides a REX-Ray Docker plugin for DigitalOcean Block Storage
Fixes #816

Codecov Report

Merging #883 into release/0.9.1 will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           release/0.9.1   #883   +/-   ##
============================================
  Coverage            8.8%   8.8%
============================================
  Files                 25     25
  Lines               2975   2975
============================================
  Hits                 262    262
  Misses              2677   2677
  Partials              36     36

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad367e2...854934e. Read the comment docs.

Ah, one more thing...  does REXRAY_BUILD_TYPE= DRIVERS=dobs need to be added to .travis.yml still? I think it does.

It does, and @cduchesne noted that in Slack in the last few hours.

So the change to the Travis file should be a part of this PR, right?


Question - Unable to get simple code akin to image loading example working.

I am on Win7 running latest Dart version and I have my dart file as below
import 'package:play_phaser/phaser.dart';

class PhaserTest
{
  void main()
  {
    Game game = new Game(800, 600, AUTO, 'output');
    State blah = new MyState();

    game.state.add('state1', blah);
    game.state.start('state1');
  }
}

//Phaser basics_1 example
class MyState extends State
{
  Sprite image;

  preload() {

    //  You can fill the preloader with as many assets as your game requires

    //  Here we are loading an image. The first parameter is the unique
    //  string by which we'll identify the image later in our code.

    //  The second parameter is the URL of the image (relative)
    game.load.image('car', 'bunny.png');

  }

  create() {

    //  This creates a simple sprite that is using our loaded image and
    //  displays it on-screen
    image = game.add.sprite(game.world.centerX, game.world.centerY, 'car');

    //  Moves the image anchor to the middle, so it centers inside the game properly
    image.anchor.set(0.5);

    image.scale.set(2);
  }

  update() {
    image.angle += 1;
  }
}

When I click my index.html to "Run in Dartium" and it loads, there is nothing but a blank tab...I don't understand, have I really done something wrong here or is something broken? :( I mean really, my code is just like your first example. To clarify, bunny.png IS in the web directory of the same project and output is a div in the body of my html.

Your main function won't be called because it is set as a class method
it should be
  void main()
  {
      new PhaserTest();
  }

  class PhaserTest
  {
      PhaserTest()
      {
          Game game = new Game(800, 600, AUTO, 'output');
          State blah = new MyState();

          game.state.add('state1', blah);
          game.state.start('state1');
      }
  }

Oh wow, really? I see, thank you.


Update Documentation: tsconfig.json Aliases

Description
Clarify in the documentation that tsconfig.json aliases are not supported in the astro.config.ts file. Suggest using relative paths for module imports in this file instead of tsconfig.json aliases.
Related issues

Closes #6003

Thanks for hopping on this one! @VoxelMC , can I ask you to review? 🙌

Thanks for hopping on this one! @VoxelMC , can I ask you to review? 🙌

I just missed it; sorry about that! I'll be on the next ones.
Thank you, gsi-kevincarrera and Otterlord, for getting this done to give Docs an issue-less New Year!


feat: introduce AtResetException for AtClient

- What I did

Introduce a new exception that can be thrown when the client identifies that a remote secondary has been reset

- Description for the changelog

feat: introduce AtResetException for AtClient

At Commons has been moved to at_libraries. will move these changes to at_libraries