Bump version 0.17

Codecov Report

Merging #1924 (6031984) into main (b150159) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1924      +/-   ##
==========================================
+ Coverage   80.95%   80.99%   +0.03%
==========================================
  Files          71       71
  Lines        7970     7970
==========================================
+ Hits         6452     6455       +3
+ Misses       1518     1515       -3


HANA Provider deployment failed with InstanceInitializationFailed error

Hello,
HANA Provide deployment failed shortly after start. In the extended log, the following explanation can be found:
The instance initialization operation failed.
Possible Causes: The operation failed due to an internal error.
Recommended Action: Retry the operation. If the issue persists, please create an issue here :- https://aka.ms/AAgc4fy.
(Code: InstanceInitializationFailed)
We're in East US 2 region, subscription id is 2fb5284c-8225-4328-8a84-602f9355da07.
Thanks,
Aleksey

Hi Aleksey,
Thank you for reaching out to us. We will look into this issue. Did you try creating HANA provider again? Is it happening the next time also?
Regards,
Neenu Vincent

Hi Neenu,
Yes I tried it several times with the same result.
Thanks,
Aleksey

Hi Aleksey,
This issue is not reproducible at our end. It would be great if you can share your email id for us to set up a call with you. Please do share the suitable time to set up the meeting as well. This would help us understand the issue more and resolve it quickly.
Thank you.
Regards,
Neenu Vincent

Hello Neenu,
Email id is<EMAIL_ADDRESS>I'm located in the US CST time zone and typically available during normal US business hours.
Regards,
Aleksey

Hi Aleksey,
Thank you for the details. We have set up a meeting. Let's discuss this in detail in the meeting.
Regards,
Neenu Vincent


iOS Performance Considerations - Read Before Using

Hello!
First off, thank you for this adapter!
I've been trialing a few mobile syncing options and different PouchDB adapters in the past few months and this was my pick until I realized there are some severe performance and memory issues on iOS.
Overall Memory
I've been experimenting with a database of 5k documents (~10 revisions), totally ~7MB, and syncing this with a CouchDB cluster and it takes quite a bit of memory compared with pouchdb-adapter-react-native-sqlite. To test this I created a clean React Native project and then synced my 5k document database. The app took about 130MB on average (in both Release and Debug mode) when using the AsyncStorage adapter vs about 45MB with the Sqlite adapter. This is a pretty severe difference.
Memory when there is other data in AsyncStorage (not related to current database)
I found that the database became increasingly slow and consumed increasing amounts of memory due to it reading ALL AsyncStorage keys from disk, even those not related to the current database. I'm using multiple databases, so this was very noticeable for me. This is not a problem with the Sqlite adapter. I will say this below again, but this is not really the fault of this adapter, it is more of a problem with how AsyncStorage's local storage implementation works on iOS. There IS a solution, if you MUST use AsyncStorage: use AsyncStorage backed by RocksDB which provides a getAllKeysWithPrefix, allowing the current database to be unaffected by other AsyncStorage data. I've created a PR to get this working with this adapter.
Slow read times
Additionally, there are performance concerns with timing. I found that the Sqlite adapter could resolve allDocs({ include_docs: true }) in about 450ms whereas AsyncStorage generally took around 8-12 seconds. I found bulkDocs and generally all of the operations to be similarly slow, even with start_key and end_key specified. They do get a bit faster when building for Release, but not faster enough to make up for the difference between this and Sqlite. Do note that the Sqlite adapter is a bit slower at writes.
TLDR
Basically, AsyncStorage on iOS is hella slow and memory inefficient and gets even worse in proportion to the amount of data you are using. Be sure to take these into consideration before using this adapter.
Again, I do not believe the performance issues with pouchdb-react-native have much to do with the adapter itself. They are mostly related to React Native's AsyncStorage and thus out of the control of this adapter. Unless the React Native team improves the performance of AsyncStorage or adds support for IndexedDB, I would recommend considering pouchdb-adapter-react-native-sqlite instead.

Thank you for such a great research! Chapeau.


App Gateway sslcert not deleted during terraform destroy (also max 100 sslcerts)

Describe the bug
We use Helm and Terraform to deploy services into AKS, it seems when we 'terraform destroy' a deployed service the sslcert data that was applied to the App Gateway is not deleted.
To Reproduce
Deployed service with an ingress controller, here is the ingress.yaml that is used as part of the deployment
kubectl get ingress -n auto-terratestjdr1 -o yaml
`apiVersion: v1
items:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
appgw.ingress.kubernetes.io/backend-protocol: https
appgw.ingress.kubernetes.io/request-timeout: "60"
appgw.ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: azure/application-gateway
creationTimestamp: "2020-08-28T14:34:10Z"
generation: 1
labels:
netsec.barracuda.com/environment: dev
netsec.barracuda.com/name: terratestjdr1
name: terratestjdr1
namespace: auto-terratestjdr1
resourceVersion: "27567478"
selfLink: /apis/extensions/v1beta1/namespaces/auto-terratestjdr1/ingresses/terratestjdr1
uid: fdd32ed1-a5b6-4fbe-95b5-016749007494
spec:
rules:

host: springboot-demo-appjdr1.dev.bnngm.azure.cudaops.com
http:
paths:

backend:
serviceName: terratestjdr1
servicePort: 8443
tls:

hosts:

springboot-demo-appjdr1.dev.bnngm.azure.cudaops.com
secretName: star-dev-bnngm-azure-cudaops-com
status:
loadBalancer:
ingress:
ip: <IP_ADDRESS>
kind: List
metadata:
resourceVersion: ""
selfLink: ""`

Verify that the cert has been installed
az network application-gateway ssl-cert list -g bnngm-dev-westeurope  --gateway-name bnngm-dev-westeurope-aks-appgw --subscription blah-blah | jq
  {     "data": null,     "etag": "W/\"cf3dcf6a-0fe4-4038-a59a-428923b3362e\"",     "id": "/subscriptions/4b7cd783-c55a-4319-a0d7-a3a68ef112b1/resourceGroups/bnngm-dev-westeurope/providers/Microsoft.Network/applicationGateways/bnngm-dev-westeurope-aks-appgw/sslCertificates/auto-terratestjdr1-star-dev-bnngm-azure-cudaops-com",     "keyVaultSecretId": null,     "name": "auto-terratestjdr1-star-dev-bnngm-azure-cudaops-com",     "password": null,     "provisioningState": "Succeeded",     "publicCertData": "FAKEDATA",     "resourceGroup": "bnngm-dev-westeurope",     "type": "Microsoft.Network/applicationGateways/sslCertificates"   }
Destroy the deployment and check for the cert again;
az network application-gateway ssl-cert list -g bnngm-dev-westeurope  --gateway-name bnngm-dev-westeurope-aks-appgw --subscription blah-blah | jq
[   {     "data": null,     "etag": "W/\"738c1667-1f66-4560-ba3e-b28ae25b4460\"",     "id": "/subscriptions/4b7cd783-c55a-4319-a0d7-a3a68ef112b1/resourceGroups/bnngm-dev-westeurope/providers/Microsoft.Network/applicationGateways/bnngm-dev-westeurope-aks-appgw/sslCertificates/auto-terratestjdr1-star-dev-bnngm-azure-cudaops-com",     "keyVaultSecretId": null,     "name": "auto-terratestjdr1-star-dev-bnngm-azure-cudaops-com",     "password": null,     "provisioningState": "Succeeded",     "publicCertData": "FAKEIT",     "resourceGroup": "bnngm-dev-westeurope",     "type": "Microsoft.Network/applicationGateways/sslCertificates"   } ]
The sslcert is not removed with the rest of the deployment.
Once we hit 100 deployments then we cannot deploy anymore.
Ingress Controller details
kubectl describe pod -n ingress-azure ingress-azure-69c7c9d66d-sjgkl
Name:         ingress-azure-69c7c9d66d-sjgkl
Namespace:    ingress-azure
Priority:     0
Node:         aks-agentpool-18118092-vmss000000/<IP_ADDRESS>
Start Time:   Mon, 27 Jul 2020 21:21:10 +0100
Labels:       aadpodidbinding=ingress-azure
app=ingress-azure
pod-template-hash=69c7c9d66d
release=ingress-azure
Annotations:  checksum/config: 92368a0b96949a0f3e010d229a45caa26fea001a744095ee3f780595cc508397
kubectl.kubernetes.io/restartedAt: 2020-07-15T11:10:45+01:00
prometheus.io/port: 8123
prometheus.io/scrape: true
Status:       Running
IP:           <IP_ADDRESS>
IPs:
IP:           <IP_ADDRESS>
Controlled By:  ReplicaSet/ingress-azure-69c7c9d66d
Containers:
ingress-azure:
Container ID:   docker://cffae75aa13e593b9d1422ac3c1aa8ea991768bd607af7b8e29fdf8467178e91
Image:          mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.2.0
Image ID:       docker-pullable://mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:de458f962eab0cd2de19d23dfeb9a0e4bc2565a38f8c45cc98a74f3cda8b940c
Port:
Host Port:
State:          Running
Started:      Mon, 27 Jul 2020 21:21:23 +0100
Ready:          True
Restart Count:  0
Liveness:       http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
Readiness:      http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
ingress-azure  ConfigMap  Optional: false
Environment:
AZURE_CLOUD_PROVIDER_LOCATION:  /etc/appgw/azure.json
AGIC_POD_NAME:                  ingress-azure-69c7c9d66d-sjgkl (v1:metadata.name)
AGIC_POD_NAMESPACE:             ingress-azure (v1:metadata.namespace)
Mounts:
/etc/appgw/azure.json from azure (rw)
/var/run/secrets/kubernetes.io/serviceaccount from ingress-azure-token-dhdlh (ro)
Conditions:
Type              Status
Initialized       True
Ready             True
ContainersReady   True
PodScheduled      True
Volumes:
azure:
Type:          HostPath (bare host directory volume)
Path:          /etc/kubernetes/azure.json
HostPathType:  File
ingress-azure-token-dhdlh:
Type:        Secret (a volume populated by a Secret)
SecretName:  ingress-azure-token-dhdlh
Optional:    false
QoS Class:       BestEffort
Node-Selectors:
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
terraform -version Terraform v0.12.29

Hi,
I'm experiencing the same issue. I don't think that it classifies as an improvement but must be treated as a bug as the ingress controller simply stops working in my environment after I've added/deleted 100 ingress definitions. I think it's important that the ingress controller cleans up after itself and when it deletes a listener it also has to delete the associated SSL certificate.
Nils

Any news on this issue. I would also agree, that this classifies as bug.
Are there any plans, when this will be fixed?

@akshaysngupta -- are you able to comment on this?


[TVM] Fix master build&CI

Master is failing due to merge. see #3489 and #3805

@tqchen @jroesch @vinx13 @icemelon9

https://github.com/dmlc/tvm/pull/3884


Update package.json

Take a pass over package.json and update for the new location.
Updates should include, but may not be limited to, the following

publisher
repository.url
bugs.url

reference https://github.com/apache/openwhisk-vscode-extension/blob/master/package.json


Register informers for v1 TaskRun and PipelineRun

This commit registers the v1 types so knative can generate informers for them.
Example error reported in PipelineRun logs (despite PipelineRun succeeding):
W0104 10:47:46.925165   85977 reflector.go:347] github.com/tektoncd/pipeline/pkg/client/informers/externalversions/factory.go:117: watch of *v1.PipelineRun ended with: an error on the server ("unable to decode an event from the watch stream: unable to decode watch event: no kind \"PipelineRun\" is registered for version \"tekton.dev/v1\" in scheme \"github.com/tektoncd/pipeline/pkg/client/clientset/versioned/scheme/register.go:32\"") has prevented the request from succeeding

/kind bug
FYI @JeromeJu @bendory
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

n/a Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed

Since this only appears in the logs, and the PipelineRun still succeeds, I'm not sure how I could write a test for this.

[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including
functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[x] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
n/a Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
bug fix: register informers for v1 TaskRun and PipelineRun

/lgtm

/retest

The third_party change seems weird, but LGTM otherwise

Concur -- especially the LICENSE deletions.

This came from ./hack/update-codegen.sh-- not sure why, but the codegen wasn't actually adding any new generated code and was also causing an indentation issue so I just reverted everything except the changes to register.go

The third_party change seems weird, but LGTM otherwise

Concur -- especially the LICENSE deletions.

This came from ./hack/update-codegen.sh-- not sure why, but the codegen wasn't actually adding any new generated code and was also causing an indentation issue so I just reverted everything except the changes to register.go

SGTM -- as a roll-forward bug-fix patch, including the minimal change set to fix the bug is a best practice anyway.

/cherry-pick release-v0.43.x


Move POC simple to this repo

We plan on deprecating the poc-simple repo:  https://github.com/Senzing/aws-cloudformation-ecs-poc-simple

Pr merged


Fix a critical bug to kill old workers on HUP

It was not killing old workers on HUP :scream:

I did not notice this because I was disabling signal sending on HUP as http://blog.livedoor.jp/sonots/archives/42826057.html


Input multiline height

Hi, how do you set input with multiline true (textarea) height?
I'm using rows attribute but didn't works.

At the moment it's not possible. The height adjusts depending on the content always in case it's a multiline Input. Maybe this behavior can be optional but I can't see the benefit. Do you want to set it fixed or just a minimum?

I see. I want to set it as minimum height, because I have requirement to put a long message in hint attribute therefore if textarea doesn't have minimum height the hint message will be broken.

Oh! Yeah that's definitively an issue. I'm trying out with some different style implementations that allows the hint element to be placed differently with CSS making the component more flexible. The textarea should auto adjust to the hint content so this is an issue. I'll try to take a look during the weekend, I'm having busy days


报错：state-refresh task failed, reason is ['NoneType' object has no attribute 'split'], 0 retry times left，请问怎么解决啊？

�2024-12-27T04:16:17.269262635Z 2024-12-27 12:16:17  [INFO    ] ---- The current repository version is v1.6.0, and the repository address is https://github.com/ARC-MX/sgcc_electricity_new.git
c2024-12-27T04:16:17.865702933Z 2024-12-27 12:16:17  [INFO    ] ---- chromium-driver version is 120
�2024-12-27T04:16:17.865742299Z 2024-12-27 12:16:17  [INFO    ] ---- enable_database_storage is false, we will not store the data to the database.
�2024-12-27T04:16:17.865829291Z 2024-12-27 12:16:17  [INFO    ] ---- The current logged-in user name is<PHONE_NUMBER>, the homeassistant address is http://homeassistant:8123/, and the program will be executed every day at 07:00.
�2024-12-27T04:16:17.874281754Z 2024-12-27 12:16:17  [INFO    ] ---- Run job now! The next run will be at 07:00 and 19:00 every day
u2024-12-27T04:16:17.896780372Z 2024-12-27 12:16:17  [INFO    ] ---- patching driver executable /usr/bin/chromedriver
[2024-12-27T04:16:33.716363788Z 2024-12-27 12:16:33  [INFO    ] ---- Webdriver initialized.
�2024-12-27T04:16:33.716451822Z 2024-12-27 12:16:33  [ERROR   ] ---- state-refresh task failed, reason is ['NoneType' object has no attribute 'split'], 4 retry times left.
[2024-12-27T04:16:49.384342416Z 2024-12-27 12:16:49  [INFO    ] ---- Webdriver initialized.
�2024-12-27T04:16:49.384424361Z 2024-12-27 12:16:49  [ERROR   ] ---- state-refresh task failed, reason is ['NoneType' object has no attribute 'split'], 3 retry times left.
[2024-12-27T04:17:05.064285447Z 2024-12-27 12:17:05  [INFO    ] ---- Webdriver initialized.
�2024-12-27T04:17:05.064353350Z 2024-12-27 12:17:05  [ERROR   ] ---- state-refresh task failed, reason is ['NoneType' object has no attribute 'split'], 2 retry times left.
[2024-12-27T04:17:20.728317866Z 2024-12-27 12:17:20  [INFO    ] ---- Webdriver initialized.
�2024-12-27T04:17:20.728503846Z 2024-12-27 12:17:20  [ERROR   ] ---- state-refresh task failed, reason is ['NoneType' object has no attribute 'split'], 1 retry times left.
[2024-12-27T04:17:36.406920484Z 2024-12-27 12:17:36  [INFO    ] ---- Webdriver initialized.
�2024-12-27T04:17:36.407016491Z 2024-12-27 12:17:36  [ERROR   ] ---- state-refresh task failed, reason is ['NoneType' object has no attribute 'split'], 0 retry times left.

.env 里面你删了什么东西


[TEMP] GitHub Pages is having issues

Before reporting an issue, please check the green ticks at https://github.com/robchahin/sso-wall-of-shame/commits/master
GitHub Pages is working through a backlog of builds due to some weekend issues, so you may be looking at an older version of the site.

Supposedly GitHub is past this now, and outstanding site bugs have been fixed, so will close this one.


How Can I start gekko?

Hello, i don't know how start gekko.
I drag the gekko.js on node.exe but it doen't run.
There are some newbie tutorial? Thank you!

I'd follow this https://gekko.wizb.it/docs/installation/installing_gekko.html

thank you so much! i Can install it, but when I try to acces for http://localhost:3000 the page reject my connection

ERR_CONNECTION_REFUSED

Have you started the gekko server and UI as per the documentation?
If yes, make sure that port 3000 is not already in use and that you do not have a firewall blocking access.

Yes node gekko --ui
I try another ports (30002,30005) with windows firewall desactivated, and all time ERR_CONECTION_REFUSED

Unfortunately at this stage Gekko is quite technical to run, I am working on a hosted version which is simpler for people without a technical background to use. Read more about that here: https://forum.gekko.wizb.it/thread-1435.html
I have to close this since this is a bug tracker and not a support forum. You can try posting in the forum here: https://forum.gekko.wizb.it/index.php

Hi! I found the problem, API an UI must have the same port!
But now, when I press star (after input API Key), if i show the console on Chrome I have this message:
Uncaught TypeError: Cannot read property 'start' of undefined
at s.gekkoConfig (build.js:16)
at Ri.get (build.js:6)
at Ri.evaluate (build.js:6)
at s.gekkoConfig (build.js:6)
at startGekko (build.js:16)
at s.n [as startGekko] (build.js:6)
at start (build.js:16)
at Proxy.n (build.js:6)
at click (build.js:16)
at HTMLAnchorElement.e (build.js:6)


Woptim/flux on corona

Adapt corona allocation to flux.

Relies on https://github.com/LLNL/radiuss-shared-ci/pull/7
@davidbeckingsale This can be merged as-is for a quick fix of Corona.


Git plugin has unknown flag --recursive on clone (gcl)

This line - Can't see the --recursive flag in the manpage; any idea what it's supposed to be there for?

--recursive is used during cloning and will clone the specified repository including all submodules

Should it not be --recurse-submodules (according to the man page)?

Right, but (from the manpage):  This is equivalent to running git submodule update --init --recursive immediately after the clone is finished.

@Billy- you're right, it's not in the man page. However when running git help clone you get the following information about the --recursive switch:
--recursive, --recurse-submodules
     After the clone is created, initialize all submodules within, using their default settings.
     This is equivalent to running git submodule update --init --recursive immediately after the
     clone is finished. This option is ignored if the cloned repository does not have a
     worktree/checkout (i.e. if any of --no-checkout/-n, --bare, or --mirror is given)

@eremit Yeah, --recurse-submodules is mentioned in the manpage and is equivalent to running git submodule update --init --recursive but that doesn't say anything about the --recursive flag being available on git clone. It's not that I didn't believe that it worked, just confused by the lack of documentation - I would have thought git would be pretty complete. Thanks for clearing that up @folixg.


Integrate into ioBroker discovery

enable ioBroker.discovery to find VBus devices in local network

Done with ioBroker.mybus v0.3.0 and ioBroker.discovery v4.2.0. Also supports ioBroker.resol discovery


Need Test account option.

-Need testing account option also here.

When send from contacts.

Is this still a feature we want to implement? I'm not seeing where the UI currently allows changing the account to send from on this screen. (@AliMeer @antuz123 @Arpanjainson)

@cakesoft-swati - when do we get this screen?

When sending through F&F list contact.


Cloudstack API end point getting failed while deploying nginx as loadbalancer on kubernetes

I have an issue with cloudstack.. i'm deploying nginx service as loadbalancer on kubernetes getting an error given below...

Events:
  Type     Reason                  Age                  From                Message
  ----     ------                  ----                 ----                -------
  Warning  SyncLoadBalancerFailed  4s (x1027 over 17h)  service-controller  Error syncing load balancer: failed to check if load balancer exists before cleanup: error retrieving load balancer rules: Get http://<IP_ADDRESS>:8080/client/api?apiKey=5yQj6eY6hAoNFrHFFNK1RFvMsioFwpcruIkBk09-GhUDcA6CO_jO1-K_Gyyoqa0l7DBxmseK70GsrGfe2opabA&command=listLoadBalancerRules&keyword=ad715cbd81ac64c079be9a71f46a7c0c&listall=true&response=json&signature=WIJuc9uYnkd5XGXmtiUcRiGWzuw%3D: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

i configured Endpoint url (endpoint.url) in Global settings using LAN IP...can anyone help me on this

@vishnuvs369
the endpoint URL should be reachable from the k8s pods.
If you can fix the URL, you can re-deploy the k8s secret by script /opt/bin/deploy-cloudstack-secret in the control node.
if not, you have to configure the load balancing rules manually.

@weizhouapache
Thanks for the update, We can either add a public ip interface to the management server and configure that IP on the endpoint url or do a NAT route on <IP_ADDRESS> IP and configure public ip on the endpoint url. Our concern is that we need our management ui to be accessed only internally. So configuring public IP on management network we will have to take care of additional security... What is your advice for best practice...

@weizhouapache
Thanks for the update, We can either add a public ip interface to the management server and configure that IP on the endpoint url or do a NAT route on <IP_ADDRESS> IP and configure public ip on the endpoint url. Our concern is that we need our management ui to be accessed only internally. So configuring public IP on management network we will have to take care of additional security... What is your advice for best practice...

However, user vms should be unable to access the management network, otherwise it is a more severe security issue.
You can add some firewall rules for the public ip of the management IP.

We are experiensing the same issue, it looks to be the same issue as 4.18 where the pages would take a very long time to load where they were pulling all data instead of what was needed. The command causing this slowness is:
https://my.url.com/client/api?apiKey=myapikeyhere&command=listVirtualMachines&listall=true&response=json&signature=mysignature
If i run this command in a browser the API takes around 5-10 minutes to load it, however is i try to run normal commands on the API such as listVirtualMachinesMetrics it is instant.

We are experiensing the same issue, it looks to be the same issue as 4.18 where the pages would take a very long time to load where they were pulling all data instead of what was needed. The command causing this slowness on our one:
https://my.url.com/client/api?apiKey=myapikeyhere&command=listVirtualMachines&listall=true&response=json&signature=mysignature
If i run this command in a browser the API takes around 5-10 minutes to load it, however is i try to run normal commands on the API such as listVirtualMachinesMetrics it is instant.

@PaX101 , your comment seems to relate to a totally different issue, does it?  Did you plane to add it to another issue?

Hi Guys,
It is not best practice to have a public interface on the management server and open port 8080 to public access. management server should have only local network access. So when configuring Kubernetes, it checks for the endpoint URL, and that URL should be accessible from the Kubernetes cluster. In such case, kubernetes VR is only has public nic and a local NIC, which will not be available to management server local network. So how is it possible to communicate a Kubernetes cluster with the management server?
Any advice will be great and helpful.

@Noelantogerorge , In your scenario I would say you need a proxy with firewall to allow your cluster to call back. And then configure the endpoint.url to be the proxy.


Log binary sizes (excl. UWP changes)

Log the sizes of binaries that the builds produce. Do not make UWP related changes.

Example query showing ingested data: https://dataexplorer.azure.com/clusters/azsdkengsys.westus2/databases/Pipelines?query=H4sIAAAAAAAAA2WQQU/DMAyF7/sV1i7dpI1VuQ+JcgWGtIm7Sd0lKE0qx9EE4scTqKJ2w7c8f89+zjMJWx0X33AxxLSAXC/YE+z3sGysR/482i9awl8HfQsn21MU7Ae4hxaFJL9XlaqV2tZqq+pqPbFP+E4u3h1DYk0No9cGspclXqyYVcXUxd2QnNtV6xxh4PBBWkb/uHzMUgZN2maETshnkgfWxkp2Jp7B/3tXpkO8RQ/xCnh1KF3g/hYreoFDcNqg9TOuSJv5Kb//mJk3dKkkaZJ17WPwnT3PjpzEH8Lq7gueAQAA
There is an issue involving some of the data not being ingested properly which we are looking into in the interim however this PR emits the data as expected.
cc @hallipr

Closing and reopening to reset CI

I think the updated approach of building metric values from the environment at runtime is much better!


programs.rofi: update to latest home-manager; add i3 swap workspaces feature

Importing changes I did downstream to the Rofi support:

It now supports home-manager's master branch, but broken on release 20.09).
Added support for swapping i3 workspaces (similar to dynamic window managers).
The list workspaces function have also been updated:

rofi/i3-support: get the list of profiles and stories before sorting
rofi/i3-support: include the list of all available stories from all profiles
rofi/i3-support: exclude the current workspace from the printed list

Do you mind if we wait for 21.05 to merge this if it's broken on home-manager's 20.09?

Do you mind if we wait for 21.05 to merge this if it's broken on home-manager's 20.09?

IMO we need to start preparing for it. What if you create a release-20.09 branch of the existing main?

What if we create a release-20.09 branch of the existing main?
Sounds good!

done!


Add generic polynomial

This is an attempt at a hacspec v2 compliant generic (uni-variate) polynomial spec.
It supports

polynomial-polynomial addition/subtraction/multiplication/division
polynomial-scalar addition/subtraction/multiplication/division
evaluation
Some convenience functions for getting degree, coefficient form, ...

As of 17-06-2023 (via docker image):

cargo-hax lint hacspec returns no errors or warnings
cargo-hax into coq gives

bash-5.2# cargo-hax into coq
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
   Compiling hacspec-pasta v1.0.0 (/work/examples/pasta)
note: Writing file "/work/examples/polynomial/out/Hacspec_pasta.v"

   Compiling hacspec-polynomial v0.1.0 (/work/examples/polynomial)
error[CE0001]: (Diagnostics.Context.Phase FunctionalizeLoops): something is not implemented yet.
               Loop without mutation?
  --> polynomial/src/polynomial.rs:46:9
   |
46 | /         for i in (1..len).rev() {
47 | |             if self.coefficients[i] != T::default() {
48 | |                 return Polynomial {
49 | |                     coefficients: self.coefficients.slice(0, i + usize::one()),
50 | |                 };
51 | |             }
52 | |         }
   | |_________^

error[CE0001]: (Diagnostics.Context.Phase CfIntoMonads): something is not implemented yet.This is discussed in issue https://github.com/hacspec/hacspec-v2/issues/96.
               Please upvote or comment this issue if you see this error message.
               TODO: Monad for loop-related control flow
   --> polynomial/src/polynomial.rs:200:9
    |
200 | /         for _ in 0..loop_upper_bound {
201 | |             if r.trim() != Polynomial::<T>::default() && r.degree() >= rhs.degree() {
202 | |                 let degree_diff = r.degree() - rhs.degree();
203 | |                 let mut t = Seq::<T>::create(degree_diff + usize::one());
...   |
210 | |             }
211 | |         }
    | |_________^

error[CE0001]: (Diagnostics.Context.Phase FunctionalizeLoops): something is not implemented yet.
               Only for loop are being functionalized for now
   --> polynomial/src/polynomial.rs:200:9
    |
200 | /         for _ in 0..loop_upper_bound {
201 | |             if r.trim() != Polynomial::<T>::default() && r.degree() >= rhs.degree() {
202 | |                 let degree_diff = r.degree() - rhs.degree();
203 | |                 let mut t = Seq::<T>::create(degree_diff + usize::one());
...   |
210 | |             }
211 | |         }
    | |_________^

error[CE0001]: (Diagnostics.Context.Phase FunctionalizeLoops): something is not implemented yet.
               Loop without mutation?
   --> polynomial/src/polynomial.rs:236:16
    |
236 |           } else {
    |  ________________^
237 | |             for i in 0..lhs.len() {
238 | |                 if lhs[i] != rhs[i] {
239 | |                     return false;
240 | |                 }
241 | |             }
242 | |         }
    | |_________^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:83:1
    |
83  | / impl<T: Numeric + NumericCopy + PartialEq> Add<Polynomial<T>> for Polynomial<T> {
84  | |     type Output = Self;
85  | |
86  | |     fn add(self, other: Self) -> Self {
...   |
106 | |     }
107 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:109:1
    |
109 | / impl<T: Numeric + NumericCopy + PartialEq> Add<T> for Polynomial<T> {
110 | |     type Output = Self;
111 | |
112 | |     fn add(self, other: T) -> Self {
...   |
116 | |     }
117 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:119:1
    |
119 | / impl<T: Numeric + NumericCopy + PartialEq> Sub<Polynomial<T>> for Polynomial<T> {
120 | |     type Output = Self;
121 | |
122 | |     fn sub(self, other: Self) -> Self {
...   |
134 | |     }
135 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:137:1
    |
137 | / impl<T: Numeric + NumericCopy + PartialEq> Sub<T> for Polynomial<T> {
138 | |     type Output = Self;
139 | |
140 | |     fn sub(self, other: T) -> Self {
...   |
144 | |     }
145 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:147:1
    |
147 | / impl<T: Numeric + NumericCopy + PartialEq> Mul<Polynomial<T>> for Polynomial<T> {
148 | |     type Output = Self;
149 | |
150 | |     fn mul(self, rhs: Self) -> Self::Output {
...   |
167 | |     }
168 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:170:1
    |
170 | / impl<T: Numeric + NumericCopy + PartialEq> Mul<T> for Polynomial<T> {
171 | |     type Output = Self;
172 | |
173 | |     fn mul(self, rhs: T) -> Self::Output {
...   |
179 | |     }
180 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:182:1
    |
182 | / impl<T: Numeric + NumericCopy + PartialEq + hacspec_lib::Div<Output = T>> Div<Polynomial<T>>
183 | |     for Polynomial<T>
184 | | {
185 | |     type Output = (Self, Self);
...   |
214 | |     }
215 | | }
    | |_^

error[CE0001]: (Diagnostics.Context.Backend Coq): something is not implemented yet.
               [ty] node typ
   --> polynomial/src/polynomial.rs:217:1
    |
217 | / impl<T: Numeric + NumericCopy + PartialEq + hacspec_lib::Div<Output = T>> Div<T> for Polynomial<T> {
218 | |     type Output = Self;
219 | |
220 | |     fn div(self, rhs: T) -> Self {
...   |
226 | |     }
227 | | }
    | |_^

note: Writing file "/work/examples/polynomial/out/Hacspec_polynomial.v"

error: could not compile `hacspec-polynomial` (lib) due to 12 previous errors

cargo-hax into fstar gives

bash-5.2# cargo-hax into fstar
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
   Compiling hacspec-pasta v1.0.0 (/work/examples/pasta)
note: Writing file "/work/examples/polynomial/out/Hacspec_pasta.fst"

   Compiling hacspec-polynomial v0.1.0 (/work/examples/polynomial)
error[CE0001]: (Diagnostics.Context.Phase FunctionalizeLoops): something is not implemented yet.
               Loop without mutation?
  --> polynomial/src/polynomial.rs:46:9
   |
46 | /         for i in (1..len).rev() {
47 | |             if self.coefficients[i] != T::default() {
48 | |                 return Polynomial {
49 | |                     coefficients: self.coefficients.slice(0, i + usize::one()),
50 | |                 };
51 | |             }
52 | |         }
   | |_________^

error[CE0001]: (Diagnostics.Context.Phase CfIntoMonads): something is not implemented yet.This is discussed in issue https://github.com/hacspec/hacspec-v2/issues/96.
               Please upvote or comment this issue if you see this error message.
               TODO: Monad for loop-related control flow
   --> polynomial/src/polynomial.rs:200:9
    |
200 | /         for _ in 0..loop_upper_bound {
201 | |             if r.trim() != Polynomial::<T>::default() && r.degree() >= rhs.degree() {
202 | |                 let degree_diff = r.degree() - rhs.degree();
203 | |                 let mut t = Seq::<T>::create(degree_diff + usize::one());
...   |
210 | |             }
211 | |         }
    | |_________^

error[CE0001]: (Diagnostics.Context.Phase FunctionalizeLoops): something is not implemented yet.
               Only for loop are being functionalized for now
   --> polynomial/src/polynomial.rs:200:9
    |
200 | /         for _ in 0..loop_upper_bound {
201 | |             if r.trim() != Polynomial::<T>::default() && r.degree() >= rhs.degree() {
202 | |                 let degree_diff = r.degree() - rhs.degree();
203 | |                 let mut t = Seq::<T>::create(degree_diff + usize::one());
...   |
210 | |             }
211 | |         }
    | |_________^

error[CE0001]: (Diagnostics.Context.Phase FunctionalizeLoops): something is not implemented yet.
               Loop without mutation?
   --> polynomial/src/polynomial.rs:236:16
    |
236 |           } else {
    |  ________________^
237 | |             for i in 0..lhs.len() {
238 | |                 if lhs[i] != rhs[i] {
239 | |                     return false;
240 | |                 }
241 | |             }
242 | |         }
    | |_________^

error[CE0001]: (Diagnostics.Context.Backend FStar): something is not implemented yet.
               anonymous impl
  --> polynomial/src/polynomial.rs:11:1
   |
11 | / impl<T: Numeric + NumericCopy + PartialEq> Polynomial<T> {
12 | |     /// Create a polynomial, defined from its coefficient form
13 | |     ///
14 | |     /// # Arguments
...  |
80 | |     }
81 | | }
   | |_^

note: Writing file "/work/examples/polynomial/out/Hacspec_polynomial.fst"

error: could not compile `hacspec-polynomial` (lib) due to 5 previous errors

@W95Psp @cmester0 are these error messages expected?
If the hacspec is correct, but the backends are lagging, then I think it could be OK to proceed with reviewing and then merging.

LGTM. I only marked a small nit.

Hello, we discussed the errors today.
For the F* backend:

the second and third errors (for _ = ...) should be fixed now (https://github.com/hacspec/hacspec-v2/pull/150)
the first and fourth errors (return) feature a return within a loop which is not currently supported. It may be easiest to simply rewrite this code to do the return outside the loop.
the fourth error involves an implicit impl (without a trait). This is currently unallowed but will be fixed soon.

So, if you make the return modification, this code will pass the toolchain soon.

Some short friendly documentation would be kind here.

@spitters
I am not sure what lines this refers to, is it in general or for impl<T: Numeric + NumericCopy + PartialEq> Add<Polynomial<T>> for Polynomial<T>?

Hello, we discussed the errors today.
For the F* backend:
* the second and third errors (`for _ = ...`) should be fixed now ([Allow any pattern on for loops (fixes #147) hacspec-v2#150](https://github.com/hacspec/hacspec-v2/pull/150))

* the first and fourth errors (`return`) feature a `return` within a loop which is not currently supported. It may be easiest to simply rewrite this code to do the `return` outside the loop.

* the fourth error involves an implicit `impl` (without a trait). This is currently unallowed but will be fixed soon.

So, if you make the return modification, this code will pass the toolchain soon.

Great! I will look into that.

I added the requested comment!
Furthermore, I wondered whether it might be useful with a function to create a polynomial using lagrange interpolation. I am not sure if that functionality is suitable for inclusion in a polynomial spec or not? (Maybe that should be separate?)

I think one could include it here, as it's too small for a separate
library. It can always be moved later.
from my phone
On Wed, Aug 9, 2023, 21:59 Rasmus T. Bjerg @.***> wrote:

I added the requested comment!
Furthermore, I wondered whether it might be useful with a function to
create a polynomial using lagrange interpolation. I am not sure if that
functionality is suitable for inclusion in a polynomial spec or not? (Maybe
that should be separate?)
—
Reply to this email directly, view it on GitHub
https://github.com/hacspec/specs/pull/5#issuecomment-1672062528, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AABTNTQM47FK4ZYKY3SI4X3XUPTYRANCNFSM6AAAAAAZKE2INU
.
You are receiving this because you were mentioned.Message ID:
@.***>


Bump golang to v1.14.2

@mumoshu  I could opened another PR to tackle upgrading to golang 1.14 if this one does not have traction anymore.
That issue delayed me yesterday while I was setting up helmfile on my local machine, cause I have go 1.14 and had to install 1.13 to do the other contribution

@renehernandez sorry for the delay, let me pick up on this PR again.

@chenrui333 I think you need to rebase master onto your branch. There are conflicts in the PR

yeah, I just did (PR is quite behind :) )

yeah, I just did (PR is quite behind :) )

Gotcha, great!!

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml.part.0: environment values file matching "env.*.yaml" does not exist in "."
error in first-pass rendering: result of "helmfile.yaml.part.0":

I think they aren't actually failures but logged error messages.
The actual failures seem to be coming from possible changes in TableWriter. Probably it changed how the empty table column is filled with spaces:

I see, let me revert those space trim stuff.

@mumoshu good to go. 🚀


No returns with integrated yelp search

We have a yelp enabled and integrated on our website on real estate listing pages.  Hundreds of them.  There are no results regardless of how wide I open map.  I used a downtown location as a test

and no results.  Current token is iE1kRnOSkgWDOAdvYDoMm0bk1FrEq_t0 .  Is it possible it needs to be changed or refreshed?  How do I get a new code?

@MVEBA : It'd helpful to see what request you're sending to the Yelp API, and what responses you're getting back --- that will show much more detail than the screenshot.

What is one example URL that isn't working?  Probably something like https://api.yelp.com/v3/businesses/search?location=....  If you're not a developer, you might need a developer's help for this step.

If that URL has "v3" in it (likely): What is your Client ID from https://www.yelp.com/developers/v3/manage_app ?  Don't share your Client Secret.

If that URL has "v2" in it (less likely): What is your Consumer Key from https://www.yelp.com/developers/v2/manage_api_keys ?  Don't share your Consumer Secret.

What actual response are you getting from the API, both HTTP status code and response body?  There's usually an error message there that will help.

          Yelp Data

          Get Directions

          Hide directions

          Street View

          Map View

Grocery
Reload
Clear

There is no data available.

Hospitals
Reload
Clear

There is no data available.

Get Directions

        <input type="hidden" id="initial_to_address" value="1 Oakdale Lane, Edgartown, MA 02539">
        From: <input type="text" id="from_address">

        To: <input type="text" id="to_address" value="1 Oakdale Lane, Edgartown, MA 02539">

          Choose a mode of transportation:

Sounds like everything is resolved, closing.

Doesn't explain everything, but good enough for today!

Turns out had another error but guessing web host issue.  No alt tags on yelp buttons.  I tried searching deep into code but no luck finding api.yelp anywhere for request(?) and nothing but not sure my search goes deep into code.


Re-work ORG statements

Many assemblers have a notion of a "real" program counter and a "logical" / "pseudo" PC.  The purpose of the latter is to allow for cases where you want to generate code for an address elsewhere in memory, usually with the expectation that the code will actually execute there after being copied or bank-switched.
The closest thing SourceGen has to a "real" program counter is the file offset, which always starts at zero.  The translation from file offset to "logical" address is performed everywhere.  Because you're essentially always in "logical PC" mode, there's no notion of starting or ending a logical block.  At the end of the block, you just set the address to wherever you need to be next.  This is fine for reconstructing the original file, but is suboptimal if the goal is to generate source code that can be modified.
For example:
.org $8000
(bunch of code)
.org $2000
(bunch of code that runs at $2000)
.org $8057
(bunch of code that runs at $8057)
If we increase the size of the code that runs at $2000, "org $8057" is no longer correct.  We'd actually want:
.org-start $8000
(bunch of code)
.org-start $2000
(bunch of code that runs at $2000)
.org-end
(bunch of code that runs at the appropriate address)
.org-end
So we would want nested "logical PC" blocks.  The outer block could be replaced by the assembler's notion of "real PC".  If the assembler doesn't support nested blocks, or doesn't have a notion of real vs. logical PC at all, we would need to "flatten" the output.
A related issue is relative positioning.  This is useful for an edge case where a block of code is mirrored in two places (perhaps by an MMU), with certain functions being called in one area, and other functions being called in the other.
For example:
.org $8000
(bunch of code for the low area)
.org-start *+$6000
(bunch of code for the high area)
.org-end
(more code for the low area)
So you might have the first bunch from $8000-8057, the middle from $8058-8086, and the last from $8087-80ff.  The middle bit uses a "logical PC" of $e058.  If you insert a byte into the first section, it moves to $e059, and the last moves to $8088.

Andy,
I fully agree with every aspect of this - very well summarized.
I wrote a little piece of dummy code below in KickAssembler format, where
the ".pseudopc { ... }" is how KickAssembler handle a block of code that is
to be expected in one location but during assembly is just placed
adjacent to the preceding code that has a different start address.
*= $8000lda #$00sta label1.pseudopc $2000{lda #$10sta label1label1: .byte
0}
lda #$20sta label1
It didn't behave as I had expected.
8000 a9 00    lda #$008002 8d 05 20 sta $20058005 a9 10    lda #$108007 8d
05 20 sta $2005800a 00       brk 800b a9 20    lda #$20800d 8d 05 20 sta
$2005
I was assuming that 8002 and 800d would point to $8005, but seemingly it
did not.
Also, KickAssembler doesn't support nested pseudopc - that gives you an
error.
/Pontus Berg
Den tis 10 aug. 2021 kl 00:39 skrev Andy McFadden @.***

:

Many assemblers have a notion of a "real" program counter and a "logical"
/ "pseudo" PC. The purpose of the latter is to allow for cases where you
want to generate code for an address elsewhere in memory, usually with the
expectation that the code will actually execute there after being copied or
bank-switched.
The closest thing SourceGen has to a "real" program counter is the file
offset, which always starts at zero. The translation from file offset to
"logical" address is performed everywhere. Because you're essentially
always in "logical PC" mode, there's no notion of starting or ending a
logical block. At the end of the block, you just set the address to
wherever you need to be next. This is fine for reconstructing the original
file, but is suboptimal if the goal is to generate source code that can be
modified.
For example:
.org $8000
(bunch of code)
.org $2000
(bunch of code that runs at $2000)
.org $8057
(bunch of code that runs at $8057)
If we increase the size of the code that runs at $2000, "org $8057" is no
longer correct. We'd actually want:
.org-start $8000
(bunch of code)
.org-start $2000
(bunch of code that runs at $2000)
.org-end
(bunch of code that runs at the appropriate address)
.org-end
So we would want nested "logical PC" blocks. The outer block could be
replaced by the assembler's notion of "real PC". If the assembler doesn't
support nested blocks, or doesn't have a notion of real vs. logical PC at
all, we would need to "flatten" the output.
A related issue is relative positioning. This is useful for an edge case
where a block of code is mirrored in two places (perhaps by an MMU), with
certain functions being called in one area, and other functions being
called in the other.
For example:
.org $8000
(bunch of code for the low area)
.org-start *+$6000
(bunch of code for the high area)
.org-end
(more code for the low area)
So you might have the first bunch from $8000-8057, the middle from
$8058-8086, and the last from $8087-80ff. The middle bit uses a "logical
PC" of $e058. If you insert a byte into the first section, it moves to
$e059, and the last moves to $8088.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fadden/6502bench/issues/107, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGZWZSRO3XBJN4JLZYNPVPLT4BKKHANCNFSM5B26CC4A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

This affects address resolution as well as code generation.  Consider a single 64KB binary divided into eight 8KB segments, one of which is switched into the 6502 address space at any given time.  In a simple scenario, with the current implementation, you would have something like this:
  .org $8000
  bank 0 stuff ($2000 bytes)
  .org $8000
  bank 1 stuff ($2000 bytes)
  .org $8000
  bank 2 stuff ($2000 bytes)
  ...

Suppose, in bank 1, there is a JSR $9f00.  SourceGen searches for a matching address within the current address grouping, then expands its search globally if it's not found.  As our sample data is currently defined, this works correctly, automatically finding address $9f00 in bank 1.
The trouble comes if bank 1 is not a single block, e.g.:
  .org $8000
  bank 0 stuff ($2000 bytes)
  .org $8000
  bank 1a stuff ($1e00 bytes)  <-- JSR is here
  .org $d000
  bank 1b stuff ($100 bytes)
  .org $9f00
  bank 1c stuff ($100 bytes)
  .org $8000
  bank 2 stuff ($2000 bytes)
  ...

If the call to $9f00 is at (say) $8100 in bank 1, the address search will fail to find a match within the same block, which only spans $8000-$9dff.  It will start searching from the start of the file, which will resolve to an offset in bank 0.
If we had nested address ranges, we could define it like this:
  .org-start $8000
  bank 0 stuff ($2000 bytes)
  .org-end
  .org-start $8000
  bank 1a stuff ($1e00 bytes)
    .org-start $d000
    bank 1b stuff ($100 bytes)
    .org-end
  bank 1c stuff ($100 bytes)
  .org-end
  .org-start $8000
  bank 2 stuff ($2000 bytes)
  ...

The search for a matching address would need to take scope into account.
(Warning: I haven't entirely thought this through.)
I think what we'd want to do is scan all of bank 1, top to bottom, giving no preference to deeper or shallower scopes.  So we'd scan "bank 1a stuff", then "bank 1b stuff", then "bank 1c stuff".  I don't think there's an advantage to a depth-first search.  We do, however, want to give everything contained within "bank 1" the first shot.
If we can't find a match at the current or lower scope, we pop up a level and retry.  We will eventually reach the top-level (full-file) scope, and just grab the first $9f00 we find.

A few notes on the proposed change...
Approach
The current UI has two modes.  If you select a single line and edit the address, you will change the address at that point and for all future offsets.  If you select a range of lines, two address overrides will be added: one at the start point, and one at the end point that resets the address to what it would have been without the first override.  The two address overrides are not connected in any way; you'd get the same effect by just setting the address in two places by hand.
We want to continue to support both modes.  Selecting a range is easy when you know the start and end, but sometimes you just want to set the start point to see how the code works from that point, without having to worry about finding the precise end.
There will be two types of address overrides.  Selecting a full range yields an override with an explicit end point, while adding one to a single line creates an override with an implicit end point.  Implicit end points are automatically placed at whichever comes first: the start of the next override, the end of a containing (parent) override, or the end of the file.
When a new override is added in a region covered by an existing override:

If the new override has an implicit end marker, things are pretty simple: the new override will be a child of whichever override includes the offset where it was created.  (It should not be possible to create a zero-length region this way, as creating it at the same offset as an explicit end marker should create a 1-byte region.)
If the new override has an explicit end marker, we need to be a bit more careful:

If the parent has an implicit end, the existing region is truncated.  The new region becomes a sibling of the existing region.
If the existing override has an explicit end marker, the new override must fit within the region defined by the parent.  The new region becomes a child of the existing region.
In both cases, the new end marker must fall within the parent's directly handled address range.  If the parent has multiple children, the new range becomes a sibling or parent of those children, and we must be sure the end range doesn't intersect them.

In either case, the new region is not allowed to exactly overlap an existing region, since that creates ambiguity: we can't know which region is the parent and which the child.
It should not be possible to create a zero-length region.

When an override is deleted, implicit end markers are re-evaluated.
It's possible to create a totally overlapping region through deletion:

Define region with explicit end (+000000 to +0000ff)
Define region with explicit end (+000080 to +0000ff)
Define region with implicit end (+000000 [to +00007f])
Delete region defined in step 2, leaving regions 1 and 3 overlapping.
Such cases will generate a warning in the message area, and result in one of the regions being ignored.  The program will not try to keep track of the order in which regions were created, or establish a priority ordering, so the results are undefined until one region is modified or removed.

This may be the correct time to introduce the idea of "no address" zones, for areas that are actually part of system headers or aren't addressable by the 6502 (such as the NES graphics tiles).  We need to address this somehow because it'll be possible to redefine the initial address map in a way that excludes the end of the file.  (The current implementation is essentially using regions with implicit ends, and the address override at offset zero is mandatory, so no part of the file can be without an address.)  These require some careful handling: unaddressable code may not be referenced from anywhere else, so we would need to prevent the code analyzer from walking into them and ignore any labels placed there.  As a first pass it might be easier to simply add a warning message identifying the presence of unaddressable regions.
UI
The existing address override edit dialog allows the user to enter the address (in hex) for the selected file offset.  There is no facility for moving an override, other than deleting and recreating it.  Overrides are removed by clearing the address field, which has focus and is selected when the dialog opens.
We want to retain this, and add a pair of radio buttons:

Explicit end at +xxyyzz
Implicit end (would be +xxyyzz)

It may be useful to make the explicit end offset editable, so that the range could be adjusted manually (though entering an offset by hand is awkward).  Transitioning from explicit to implicit requires evaluating where the implicit end would fall.  Transitioning from implicit to explicit just takes the current implicit value and records it explicitly.
If you select a range of addresses that begins with an address override, then when you open the dialog the "implicit end" button will be disabled, and the "explicit end" button will look like:

Explicit end at +xxyyzz (was {explicit,implicit} +aabbcc)

The "was" part only appears if the selection doesn't match the original.  This allows you to resize an explicit region, or convert an implicit region to explicit using an end point other than the current explicit end.
We need to add an additional line that indicates the new region is being created inside an existing region (show offset/addr).  Another line will appear if the new region is becoming the parent of one or more existing regions.
We can handle non-addressable regions by having the user type "NA" in the address box.  This way the override can be changed or deleted easily without the need for additional controls.
The "edit address override" option will not be available if the line selection is discontiguous.  (Alternatively, we can just pay attention to the first and last selected lines.)
We can add a "relative" checkbox that causes the .ORG to be displayed as "* + offset" rather than an absolute value.  This is only important for assembly source code generation.
Coincident Start Points
It's useful to consider the value of allowing multiple nested regions to start at the same offset.  The overall UI could be simplified a bit if we only allowed one address override per offset.  This turns out to be problematic however.
Suppose the map is:
  +----------
  | region 1
  | +----------
  | | region 2
  | +----------
  | region 1 continued
  | +----------
  | | region 3
  | +----------
  | region 1 continued
  +----------

This is obvious: create region 1 spanning a large area, then create regions 2 and 3 inside.  (Also allowed: create regions 2 and 3, then create region 1 to wrap them.)
Suppose instead region 1 and region 2 start at the same offset:
  +----------
  | +----------
  | | region 2
  | +----------
  | region 1
  | +----------
  | | region 3
  | +----------
  | region 1 continued
  +----------

This isn't possible if we allow only one region per file offset, because both region 1 and 2 start at zero.  Instead we'd have to do something like this:
  +----------
  | region 2
  + ----------
  | region 1
  | +----------
  | | region 3
  | +----------
  | region 1 continued
  +----------

This doesn't work properly, however, because code in region 1 will no longer correctly resolve addresses to locations in region 2, since it's now a sibling instead of a child.
We need to be able to create regions with overlapping start points.  In the on-screen code list, we will need to order the .ORG lines so that the ones appearing first have the end points that appear last, presenting a properly nested view.
Map Visualization
Nesting can be confusing, especially in a larger project.  There needs to be a way to display the address table that makes it easy to understand which ranges are nested.
The simplest implementation would be an ASCII-art map like those above.  A nicer view would show a graphical display with approximately correct proportions and color-coding.
Assorted Thoughts
Assorted thoughts:

If we allow regions to be deleted from the code list (select single line, hit Del), it should be possible to delete them by selecting the start or end marker.
Attempting to create a range that straddles the start or end of another range is illegal.  When the project file is loaded, straddles will be identified and ignored with a warning.
Implicit end markers are placed before the start of the next region, or the end of the parent region, whichever comes first.
We don't need to store an explicit hierarchy in the project.    We probably want to maintain a tree internally so that, when converting an address to an offset, we can identify the proper scope quickly (especially if we have to move outward.
New projects will have an address override at offset zero, with an implicit end point.  The initial address value is platform-specific.
The 65816 has some unique issues.  There's probably no reason not to allow code to spill into the next bank, though in practice it doesn't work that way.
Start/end markers in the on-screen code list will have comments identifying the offset of the pairing, and an indication of whether the end is fixed or floating.
Zero-length regions should be prevented, but don't actually break anything.
Assemblers don't expect labels on .ORG lines, so there's no value in supporting one.
Currently all offsets have an address.  It's possible "no-address" regions may have unexpected side effects.

Implementation
The address override table entries currently have "offset", "address", and "length", though the length entry is currently just an optimization (computed as the difference between the current and next offset), and is not included in the project file.  The length entry would become part of the file, and would hold either the block length for explicit ends or -1 for implicit ends.
The "no-address" zones would be output as address=-1.  A new flag would need to be stored to indicate "relative" .ORGs.
Existing project files should just work, with all existing address overrides handled as having implicit ends.  Opening a newer project file with an older version will succeed, though the explicit ends will be lost, potentially resulting in incorrect behavior when a region doesn't end when it should.  The project file version number should be increased to cause a warning at load time.
Some things to do:

Rewrite CommonUtil.AddressMap to support nesting and explicit ends.
Update address override edit dialog.
Update AddressMap load/store in project file (bump version).
Add offset-to-address map display.
Update asm source generators to handle new scheme.  For assemblers without nested "logical PC" handling we can output flattened code, for others we want to match the project.
"No-address stuff": update code and data analyzers to ignore them.  Update code list and asm generators to display them correctly.
"Relative" ORGs: update code list and asm source generators to handle them.
Add some convoluted regression tests to exercise various valid and invalid cases.

Address-to-Offset
The existing method takes the file offset of the reference (the "source") and the target address as arguments.  The source offset is what allows the existing code to scan the nearby region before looking in the file at large.  This part of the API shouldn't have to change.
We will need to generate a search tree internally.  It will need to split blocks around children.  For example, given:
  +000000 addr-start=$2000
  +000100 addr-start=$8000
  +000200 addr-end         ;+000100 explicit
  +000400 addr-end         ;+000000 implicit

The mapping is:
  $2000-20ff: +000000
  $2200-23ff: +000200
  $8000-80ff: +000100

We cannot simply say "$2000-23ff: +000000" because there's a hole at $2100-21ff.  If a bit of code referenced $2100 we would point it at $8000, which is incorrect.
We can traverse the tree in various ways.  The order matters when multiple ranges cover the same addresses.  The exact choice is probably not important, because no matter what we choose we are likely to get it wrong in one case or another.  What's most important is to make a choice and stick with it.
I suspect the most efficient approach will be for each tree node to have a list of blocks at the current level (as noted above), and a list of pointers to children.  We search all blocks at the current level, then descend into the children in order of start offset.  (All of our sibling children must have a unique start offset, because if two blocks start at the same offset then one must be a child of the other.)  We finish the descent into one child before visiting the next.
If we fail to find a match for the target address at the current depth, we back up to the parent and try again.  (We set a boolean "already checked" flag in the current node before moving up to improve efficiency.)
Asm Source Generation
Some assemblers (e.g. 64tass) support nested "logical" address ranges.  Others may support only a global "physical" PC with a single "logical" value, or just have a single flat space with .ORGs that set the address for all remaining code.
Generating the best possible code for each assembler may require some effort, but it should always be possible to fully flatten the space.

Just a question that is possibly a handling thing, which I believe lives as an issue in the same domain as this topic;
            ldy   #$00            ;2

L09B4      lda   $0a12,y         ;4+
sta   L02A7,y         ;5
iny                   ;2
cpy   #$59            ;2
bne   L09B4           ;2+
jmp   L02A7           ;3
That is a small copy routine that copies data from a source location ($0a12) to a destination location ($02a7). Data copied is a routine to be executed at the new location.
When I load the file, the source is given a label and the destination is absolute. When I set the address of the source block to the destination address then the source becomes absolute.
How can I make both relative? They are sort of fighting over the same label ...
In Kickassembler code, it would be something like
            ldy   #$00

!:              lda   source,y
sta   destination,y
iny
cpy   #sourceend-source
bne !-
jmp destination
source:
.pseudopc $02a7
{
destination:  // insert the routine
}
sourceend:

This has been on the "TO DO" wiki for a little while (https://github.com/fadden/6502bench/wiki/TO-DO-List ... search for "Some programs have short segments").  I don't have a great answer for it.
You can actually see the code relocation issue in the very first tutorial... this:
1002: b9 17 10     L1002    lda     L1017,y
1005: 99 00 20              sta     $2000,y
[...]
1014: 4c 00 20     L1014    jmp     $2000

1017: ad           L1017    .dd1    $ad

changes to this:
1002: b9 17 10     L1002    lda     $1017,y
1005: 99 00 20              sta     L2000,y
[...]
1014: 4c 00 20     L1014    jmp     L2000

when you set the address at the target location to $2000.  The STA becomes symbolic but the LDA becomes numeric.
The fundamental problem is that the byte starting at +000017 has two addresses, one "before" and one "after".  You can't give two addresses to the assembler, so you have to pick one that works automatically and then fiddle with references to the other.
One of the addresses (usually the "from" address, i.e. $1017) is effectively outside the file bounds, which makes it awkward to deal with.  You could create a Project Symbol for it, but that would hard-code the address, which is undesirable.  You could make the "from" address in the LDA instruction relative to the JMP instruction that comes right before the block that's moved:
1002: b9 17 10     L1002    lda     jump_point+3,y
1005: 99 00 20              sta     L2000,y
1008: 88                    dey
1009: 30 09                 bmi     jump_point
100b: 10 f5                 bpl     L1002

100d: 00                    .dd1    $00
100e: 68 65 6c 6c+          .str    “hello!”

1014: 4c 00 20     jump_point jmp   L2000

                            .org    $2000
2000: ad 00 30     L2000    lda     $3000

This isn't ideal, because you want the actual "from" address, not something that happens to be nearby.  For this situation we might want to generate a label right before the .ORG, i.e. "before_label = *".  The numeric reference resolver would need to add such labels to the set to resolve them automatically.
Your example also shows another thing that SourceGen doesn't handle: expressions with multiple symbols (cpy #sourceend-source).  This situation comes up often enough that it might be worth figuring out how to support simple two-symbol addition/subtraction.

I'm always worried that what I point out is already in the manual, or I
overlooked the obvious. Great that this wasn't the case and that you are
aware of the issue at hand.
I sent my example how I would have implemented it in source and I guess
something like that is what I would be my suggested target in a reassembler
as well.
Basically setting a label on a blank line before the org would give a
reference to the source and then a label on the first line after the org
would give the destination. That at least beats hardcoded valiables. That
is how I do it patching a routine that ai copy and launch. Sometimes I do a
calculation of an offset in the routine and patch that (source +
(patchlabel - destination)).
Evaluation of parameters would also be very nice. I always calculate
lengths of filenames and size of blocks to copy like that. And you see my
reference to patching logical pc routines above
Den lör 11 sep. 2021 18:24Andy McFadden @.***> skrev:

This has been on the "TO DO" wiki for a little while (
https://github.com/fadden/6502bench/wiki/TO-DO-List ... search for "Some
programs have short segments"). I don't have a great answer for it.
You can actually see the code relocation issue in the very first
tutorial... this:
1002: b9 17 10     L1002    lda     L1017,y
1005: 99 00 20              sta     $2000,y
[...]
1014: 4c 00 20     L1014    jmp     $2000
1017: ad           L1017    .dd1    $ad
changes to this:
1002: b9 17 10     L1002    lda     $1017,y
1005: 99 00 20              sta     L2000,y
[...]
1014: 4c 00 20     L1014    jmp     L2000
when you set the address at the target location to $2000. The STA becomes
symbolic but the LDA becomes numeric.
The fundamental problem is that the byte starting at +000017 has two
addresses, one "before" and one "after". You can't give two addresses to
the assembler, so you have to pick one that works automatically and then
fiddle with references to the other.
One of the addresses (usually the "from" address, i.e. $1017) is
effectively outside the file bounds, which makes it awkward to deal with.
You could create a Project Symbol for it, but that would hard-code the
address, which is undesirable. You could make the "from" address in the LDA
instruction relative to the JMP instruction that comes right before the
block that's moved:
1002: b9 17 10     L1002    lda     jump_point+3,y
1005: 99 00 20              sta     L2000,y
1008: 88                    dey
1009: 30 09                 bmi     jump_point
100b: 10 f5                 bpl     L1002
100d: 00                    .dd1    $00
100e: 68 65 6c 6c+          .str    “hello!”
1014: 4c 00 20     jump_point jmp   L2000
                        .org    $2000

2000: ad 00 30     L2000    lda     $3000
This isn't ideal, because you want the actual "from" address, not
something that happens to be nearby. For this situation we might want to
generate a label right before the .ORG, i.e. "before_label = *". The
numeric reference resolver would need to add such labels to the set to
resolve them automatically.
Your example also shows another thing that SourceGen doesn't handle:
expressions with multiple symbols (cpy #sourceend-source). This situation
comes up often enough that it might be worth figuring out how to support
simple two-symbol addition/subtraction.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fadden/6502bench/issues/107#issuecomment-917433664,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGZWZSTXW2L7KR37TEWYKHTUBN7CZANCNFSM5B26CC4A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

With regard to the implementation of the function that maps an address to an offset...
When we see something like LDA $1234, we need to figure out if the target address is inside the file, and if so, at what offset.  The algorithm requires knowing the offset at which the LDA appears, because files with overlapping regions may have multiple locations with the address $1234.  The question is how to manage this when the regions are arranged in a tree.
Suppose we have a 16K file divided into two 8K chunks:
+- +000000 START ($8000)
|  $8000 - $9fff (length=$2000/8192 bytes)
+- +002000 END (now -NA-)

+- +002000 START ($8000)
|  $8000 - $80ff (length=$0100/256 bytes)
| +- +002100 START ($e100)
| |  $e100 - $e2ff (length=$0200/512 bytes)
| +- +002300 END (now $8300)
|  $8300 - $90ff (length=$0e00/3584 bytes)
| +- +003100 START ($f100)
| |  $f100 - $f2ff (length=$0200/512 bytes)
| +- +003300 END (now $9300)
|  $9300 - $9fff (length=$0d00/3328 bytes)
+- +004000 END (now -NA-)

The second chunk has a couple of relocated sections.
If we have LDA $8001 at offset +000005, it will resolve to +0000001.  If we have that same instruction at offset +002005, it will resolve to +002001.  Easy.
If the LDA appears at offset +002105, at address $e105, it will also resolve to +002001, because when we don't find a match for the address in region 1.1, we back out a step and check region 1.0.
Suppose instead we have LDA $8101.  If it's in the first chunk it's easy to handle, but if it's anywhere in the second chunk there is no such address since that's in the hole that got mapped to $e100.  We could resolve it to an offset in the first chunk, but that seems contrary to the intent of an overlapping address space: the other chunk simply doesn't exist while the current chunk is present.
Suppose we have LDA $9301 at +002105 (at $e105).  The reference is in the $e100 chunk, but the address range isn't in that chunk.  So we back out one level, and see that the parent doesn't cover it, but one of its other children does.  We resolve it to the $9300 chunk.
The basic algorithm is something like this:

Each node in the tree has a file offset, a start address, and a length.
Descend into the tree to find the region that includes the source reference offset.  Start there.
Loop until top of tree hit:

Check the current node to see if it contains the address.  If it does, we're done.
Move up to the parent node.
Descend into each sibling, in order of increasing file offset, to see if we can find a node that contains the address.  Because every node holds an arbitrary address range, we need to search them all.

The tricky part has to do with the definition of "contains".  If we trivially test the second chunk for address $8105, it matches on the main chunk, since it falls inside a block with addr=$8000, length=$2000.  We need to take into account that there are pieces of the block that don't exist.  We know that the "holes" correspond to the child nodes, so we can examine the offset/length values of the children to see if the address in question is excluded.
The further complication is knowing how far up the tree to travel.  Ideally we wouldn't match a reference to address $8105 in the second chunk to an offset in the first chunk, since those are supposed to be mutually exclusive.  However, it's entirely possible that $8100-82ff really is supposed to be defined elsewhere in the file.  The fact that something in the second chunk is referring to it at all is the source of the problem (but consider issue #109).
We could try to apply a heuristic here, e.g. if a sibling has the same address and length if one we've already scanned then we don't descend into it, but there's probably a situation where that's the wrong answer.  I think we have to search the entire tree, and any references to addresses in "holes" will potentially resolve to a distant offset, and need to be fixed up by hand.
We could add an explicit "no overlap" flag that the user would set, but at some point the UI will transition from "complex" to "incomprehensible".
If performance becomes an issue we can create a "flattened" structure that maps address ranges to regions.  The trick is that we need a separate map for every start node.  We can generate them individually, on first use.

A minor issue remains, which I discovered while playing with Metroid (https://www.metroid-database.com/source-code/).  The memory map for the cartridge looks like this:
[...]
+000010  +- start 'L8000_0'
         |  $8000 - $bfff  length=16384 ($4000)
+00400f  +- end

+004010  +- start
         |  $8000 - $bfff  length=16384 ($4000)
+00800f  +- end
[...]

+01c010  +- start 'RandomNumbers'
         |  $c000 - $ffff  length=16384 ($4000)
+02000f  +- end

It has multiple banks switched into memory from $8000-bfff, and a permanently-resident segment from $c000-ffff.  One of the banks is graphics data, the others are code.  Most of the code banks have redundant sections, e.g. an identical jump table at the very start.
A couple bits of code (at $dd18 and $fb7b) call $80b0.  In most of the segments, that's a short routine, but in the very first segment it ends up in the middle of an instruction:
80ae: c8                           iny
80af: f0                           beq ▼ L80BE
80b0: 0d           L80B0_3         ora ▼ $40c0
80b1: c0 40                        cpy   #$40

I set the labels on the JSRs to point to a $80b0 instance in a different bank, which got rid of the auto-label, but didn't solve the actual problem:
80ae: c8                           iny
80af: f0                           beq ▼ L80BE
80b0: 0d                           ora ▼ $40c0
80b1: c0 40                        cpy   #$40

The problem is that the code analyzer doesn't know about symbols, so putting a label on the JSRs doesn't affect it.  It just knows that it needs to find $80b0, and it knows that it exists in the first segment, so it goes there.
There isn't a "correct" solution here.  The call could legitimately go to 5 different segments.  The problem is that the first segment doesn't happen to be one of them.  The earlier problems with the other 8x8KB ROM had a single correct answer, and defining the address region hierarchy correctly made everything work.
This doesn't result in incorrect code -- the JSR still goes to the same place -- but we could end up generating funny-looking code if the code generator splits the instruction into individual bytes unnecessarily.


Update Composer dependencies (2023-11-21-00-10)

Visual regression test passed!
View the visual regression test report


v5 SplitAggregated Streams doesn't work if you add a findReplaceFilter

I have a simple pipeline that does raw streaming, the output stream has splitAggregatedStreams=true.
The pipeline structure is source->streamAppender, when the input is an aggregated stream, then the output is split out as expected.
If I add a findReplaceFilter between Source and StreamAppender, then the splitAggregatedStreams functionality doesn't appear to work.
With the findReplaceFIlter in place I just get the original aggregated stream as output.

Fixed in v6.0-beta.9

Fixed in v6.0-beta.9


Add tests

Here's the nice example on how this can be achieved: https://github.com/he9qi/ueberauth_weibo/tree/master/test

Added tests: https://github.com/sobolevn/ueberauth_vk/commit/f634fc58d00ff2aa528f9ab3f342d3f5ec386a4e
Current coverage is 86%

https://github.com/shinyscorpion/ueberauth_steam/blob/master/test/ueberauth/strategy/steam_test.exs


Msource improper resampling

I had generated and verified files with a single 100 Hz tone at various sample rates (1 kHz sample rate, 10 kHz, etc up to 100 MHz sample rate).  I loaded these files into memory and sampling them using the user source in the msource object.  I notice when I set the user source bandwidth equal to this original sample rate divided by my overall sample rate for the output, I correctly get the files re-sampled so that they all look to be the same frequency.  That is until my overall normalized bandwidth gets too small (I start noticing issues around ~<0.001 bandwidth for the user source).  As the bandwidth for the output data keeps going up (effectively making the BW of each user source smaller) I notice the timing distortion becomes worse and eventually it just doesn't resample at all and the frequencies are clearly different.
I recreated the issue with minimal changes to the code structure in msource, I am attaching the file sources I used and the example code which displays the issue nicely.  Anyone can by eye tell that the frequency is off significantly by counting the peaks/troughs.  Looking at resampler rate vs actual resampler rate doesn't seem to account for the issue alone.
The person verifying this may need to modify the file path macros depending on where they put these files.  If you run the example for the different USE_FILE macro values (1-3) and run the resulting MATLAB script you should see the same plots.  I am also including the plots here for reference.  A warning that MATLAB is extremely slow using such large source files (48000 complex data points in it since I wanted to not disturb the code structure as much as possible).  I also notice it seems like when this happens there are distortions going beyond the initial 9600 FIR filter taps in the samples, seen when comparing the sharp cutoff at higher sample rates at 9600 samples.
Test_Data.zip

Seems to be related to the P value not being allowed to go below 2 (required for the fir polyphase filter bank channelizer #2).  It may require resampling beforehand before being able to handle such narrowband signal inputs.

It seems there are skews in the frequency output even before the 1/600 bandwidth lower limit, I've had the best results sticking to signals of bandwidth above 0.05.

I guess I'm not exactly sure what the issue is. The transient visible at the beginning of the burst is directly related to the bandwidth of the filter applied to the signal (essentially the bandwidth parameter given to the msource object). Is part of the issue the delay that's imposed by the channelizer?

In this case the transient is an issue but not unexpected, rather the frequency being incorrect by a significant amount is the most pressing concern.  Something about the bandwidth not being compensated for correctly at small bandwidths.  If I resample a waveform before putting it to msource, and limit msource bandwidth to not go below 0.05, I get fairly good results.

Can you provide a simple example that reproduces this?

Woops I thought I had included this the example source code in that zip, guess not...  Here is the code I ran.  Combining the MSource with an extra arbitrary resampler beforehand has given me fairly good results, but then needs an extra stage for very narrowband files.
test_code.zip

Ok, I'll take a look at it. I threw something together quickly, and I'm definitely seeing some odd behavior.
https://gist.github.com/jgaeddert/7215bdc268e0d32e24fed0b6f7f26e67


[BUG] updating pivpn does not create new wireguard config and script folders

In raising this issue, I confirm the following:

[x] I have read and understood the contributors guide.
[x] The issue I am reporting can be replicated.
[x] The issue I am reporting can be is directly related to the pivpn installer script.
[x] The issue I am reporting isn't a duplicate (see FAQs, closed issues, and open issues).

Describe the bug
after updating from an install prior to may 19 to fix the new commit that fixed the removal of wireguard clients, it seems the paths for config, vars file and scripts changed. the update script did not account for that. and some commands fail since it can't find the files in the new path since they don't exist.
no /etc/pivpn/wireguard folder, no /opt/pivpn/wireguard folder
To Reproduce
Steps to reproduce the behavior:

do an install based on a commit prior for may 20
run pivpn -up
error: /opt/update.sh: line 108: syntax error near unexpected token `else'
run pivpn -r
error: ::: Missing setup vars file!

Expected behavior
normal ability to remove, add clients, update, list clients
Server:

VPN Protocol: wireguard
OS: ubuntu 18.04
Version: latest commit June 9, 2020

Additional context
Add any other context about the problem here.
I was able to manually fix this by creating the missing folders and copying the expected conf and scripts there after checking the existing scripts that were updated (/usr/loca/bin/pivpn and /opt/pivpn/removeCONF.sh):
mkdir /etc/pivpn/wireguard
cp /etc/pivpn/setupVars.conf /etc/pivpn/wireguard/
mkdir /opt/pivpn/wireguard
git clone https://github.com/pivpn/pivpn /opt/.pivpn
cp -R /opt/.pivpn/scripts/* /opt/pivpn/

it seems the paths for config, vars file and scripts changed. the update script did not account for that. and some commands fail since it can't find the files in the new path since they don't exist.

That's a limitation of the current update script, in fact for this reasons updating is disabled for new installs.
Paths changed to accommodate the dual mode feature and just cloning the repository is not enough, you also need to detect if those changes would break previous installations.
For example the fix you are referring to also changes the way clients are added, which means you need to update the wg0.conf as well according to the new format.

I've updated wg0.conf to match the new format. is there a list of changes to be aware of or I just need to read through all the commits since then?

Main changes here. Some fixes here. Other changes in the commit log, however most of them don't apply to previous installations.

Sorry, I just ran into this myself. So, if my pivpn installation borked? Should I re-install everything?

Sorry, I just ran into this myself. So, is my pivpn installation borked? Should I re-install everything?

Wondering the same. Everything including pivpn -u is returning ::: Missing setup vars file!

Sorry, I just ran into this myself. So, is my pivpn installation borked? Should I re-install everything?

Wondering the same. Everything including pivpn -u is returning ::: Missing setup vars file!

You can fix this with the commands in my original post.

Sorry, I just ran into this myself. So, is my pivpn installation borked? Should I re-install everything?

Wondering the same. Everything including pivpn -u is returning ::: Missing setup vars file!

You can fix this with the commands in my original post.

Okay, tried that out and still getting the same message. I'm using OpenVPN setup on my instance so I will try subbing out that for wire guard.

You can also revert to a commit that has the older path (replace openvpn with wireguard if needed):
sudo -s
git clone https://github.com/pivpn/pivpn.git /etc/.pivpn
git checkout 4e8d4dfd8ef01e49f3137d5fc5a2afa14a465e47
cp /etc/.pivpn/scripts/*.sh /opt/pivpn/
cp /etc/.pivpn/scripts/openvpn/*.sh /opt/pivpn/
cp /etc/.pivpn/scripts/openvpn/bash-completion /etc/bash_completion.d/
exit

Works for me now. Thanks everyone.

Since there is no support for migrating from old format, and the only fix is manually or reinstall, I am closing this.

Since there is no support for migrating from old format, and the only fix is manually or reinstall, I am closing this.

any instruction on how to do a clean uninstall at least?

delete pivpn folders (backup your configs if you want to re-use without re-issuing new keys) and re-run the installer. if you want to restore existing configs then that will probably have to be done manually in the text files


Template  in application-authentication.xml

The <CallbackFactory> config is an optional config and it should be templated in a way that the tag should only appear in the application-authentication.xml if it is defined only.
<CallbackFactory> should be defined inside the <Extensions> tag
ex:
<Extensions>
        <RequestCoordinator>org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultRequestCoordinator</RequestCoordinator>
        <AuthenticationRequestHandler>org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultAuthenticationRequestHandler</AuthenticationRequestHandler>

        ...
        <CallbackFactory>org.wso2.carbon.identity.custom.callback.userstore.CustomUserStoreOrderCallbackFactory</CallbackFactory>
</Extensions>

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Dropdown login form

Hello, I came with a problem while trying to create dropdown login form.The problem is when I click any of input boxes it minimizes immediately. I searched in documentation, but found only one example (with class="ui input"), but that doesn't work for me.Here's my code:

                     Log In

                        <form class="ui form item">

                                Username
                                <input type="text" placeholder="Username">

                                Password
                                <input type="password" placeholder="Password">

                                    <input type="checkbox" tabindex="0" class="hidden">
                                    Remember password and username

                            <button class="ui button" type="submit">Sign In</button>
                        </form>

and js:
$('.ui.dropdown')                 .dropdown()               ;
Maybe someone have some ideas how to fix it ?
P.s. Thanks for help

Hi, thanks for posting, unfortunately although your question is valid, this is a usage question, and not appropriate for posting on GitHub Issues.
To help make this board manageable for maintainers to effectively work on actionable changes to the library, we've limited posts on this board to specific bug reports or feature requests. To see how we differentiate between the two, please check out our Contributing Guide.
To help get answers to your question (or get feedback that might allow you to repost this issue) you might want to check out one of our other support resources.

StackOverflow
ProjectTalk
Our Gitter chatroom.


Cognito Permissions to look down which calls users can view

Is there any way of using Cognito or any other method as part of the PCA solution to lock down which calls certain users can view in the dashboard?
Is there any sort of general permissions that can be applied to users?

Hi hyde-mattholgate,
Currently there is not a way to do this, however the List lambda function in the pca-ui could potentially filter the DynamoDB query based on the authorizer claims. Do you have any more details on how you would like the call list to be filtered?
Chris


[Feature Request]: Build Script: Write a script to install the required dependencies

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Creating a script(install.sh) to install the prerequisites required for the build & publish package
Record

[X] I agree to follow this project's Code of Conduct
[X] I want to work on this issue
[X] I'm willing to provide further clarification or assistance if needed.

Full Name
Yash K. Saini
Participant Role
GSSoC-Ext

@UTSAVS26 kindly assign me this issue

@yashksaini-coder you can raise PRs after raising issues I will add labels and assign you on PR directly. And complete all those raised issues that you raised.

@yashksaini-coder you can raise PRs after raising issues I will add labels and assign you on PR directly. And complete all those raised issues that you raised.

Yes I will,


What's the difference between Training scripts and Pretrain scripts?

also, What's the difference between the scripts in models/base and models/cec?

Please refer to the answer in #14


use the cache for agent config to prevent crashing when cmsweb is down

@amaltaro, Alan please review

test this please

test this please

This test works just fine on my VM. Triggering it one last time.

test this please

And that's a different test, so I assume we are Okay with this PR.


provide option to select pattern on no video

During the recording of files on unstable analog sources, bmdcapture will insert a pattern if there is no video (since 5a844f98c2c6c6387bdaa2186c6ac4c8b7cf839a). In 8 bit it is a color bar pattern and in 10 bit it is uyvy422 data rendered in v210 so a green/purple partial frame pattern. When working with unstable analog sources there are times when no frame data is received in the middle of a recording and the pattern is used. I propose that the pattern be selectable with color bars as default but including an option for black. I think an inserted occasional black frame would be less distracting to the current way bmdcapture handles unstable signals. I'm attaching two examples of the current state.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/21006611-provide-option-to-select-pattern-on-no-video?utm_campaign=plugin&utm_content=tracker%2F256764&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F256764&utm_medium=issues&utm_source=github).

That is easier to do, I'm pondering about adding a loader later to get any
image. I'll bake something tonight, can you test it and report back?
On Mon, Jun 8, 2015 at 6:35 PM, Dave Rice<EMAIL_ADDRESS>wrote:

During the recording of files on unstable analog sources, bmdcapture will
insert a pattern if there is no video (since 5a844f9
https://github.com/lu-zero/bmdtools/commit/5a844f98c2c6c6387bdaa2186c6ac4c8b7cf839a).
In 8 bit it is a color bar pattern and in 10 bit it is uyvy422 data
rendered in v210 so a green/purple partial frame pattern. When working with
unstable analog sources there are times when no frame data is received in
the middle of a recording and the pattern is used. I propose that the
pattern be selectable with color bars as default but including an option
for black. I think an inserted occasional black frame would be less
distracting to the current way bmdcapture handles unstable signals. I'm
attaching two examples of the current state.
[image: bars1]
https://cloud.githubusercontent.com/assets/762604/8039895/afd0f560-0dda-11e5-9865-2336485724cd.gif
[image: bars2]
https://cloud.githubusercontent.com/assets/762604/8039896/afd32e48-0dda-11e5-8d63-6ed1f8e9f9f4.gif
—
Reply to this email directly or view it on GitHub
https://github.com/lu-zero/bmdtools/issues/55.

I made some progress at replicated your color bars in v210 but got stuck at padding the line to a multiple of 128. At any rate IMHO simply getting black instead of the color bars is the greater need. I am happy to test.

It works for me. Also could also set a new github release?

Let me shuffle some commits, I'm in the middle of adding support of vanc data now that eventually I have access to content using it.
And I have another toy you will like a lot brewing, but probably I'll need your help (more on private email).


License

Hi,
Sorry for using issues to ask you a question.
Are you using GNU license intentionally? It would be great if the project's license was more permissive. Please see more info on the matter.
Best,
Alexander

That was a mistake. It probably should have been LGPL instead of GPL. But I can as well change it to MIT. Simple and very permissive.

OK, good to know. Looking forward to seeing it released :)
Thanks!

I assure you that you can use the current release "0.1" with the MIT license. You can point to this task.
Since there are no changes right now, I won't release a 0.2 at this time. Looking forward to contributions ;-)