Added Support for Bro language

Added support for Bro language.

Added support for Bro language.  Please let me know if you need any additions or changes.

Please ignore the change I made to examples.js -- I did that for local testing purposes.

Hi @wayward710
Thanks for your contribution! Bro is a really interesting language.
Here are some things I've noticed about your PR:

There are some minor whitespace issues in your PR. You mixed tabs and spaces for indentation. You should only use tabs.

Please ignore the change I made to examples.js -- I did that for local testing purposes.

Well you have to remove it from the PR eventually :smile:

You overuse the array syntax for tokens a little bit. The performance is actually quite bad, so it is better to avoid it if possible. There are other issues too. For example the order of the array elements is important. The first elements are matched first and can prevent longer and better matches from other elements down the line.

You should use non-capturing groups whereever possible \b(?:keyword1|keyword2)\b

On the example page you use the module keyword in an example, but it is missing in the language definition.

I don't know anything about Bro, but in your example, the redef builtin seems to be used without the leading ampersand. Is the ampersand part of the builtin keyword or more like an operator? On this page it seems to be highlighted as an operator https://www.bro.org/sphinx/scripting/index.html

I am away for the weekend, so I cannot answer any questions until Monday...

Thanks for your feedback!  I'm also traveling this weekend, so it may be a few days before I can work on the changes.

OK, I think I fixed the issues that you pointed out.  Please let me know if anything else needs to be changed.

Hi @wayward710,
Thanks! It looks really nice now.
There are only a few small issues. Other than that I think it is ready to be merged.

You still have some whitespace issues. I know it is not all that important, but it just looks unprofessional:

You should configure your editor to display whitespace characters and automatically remove trailing whitespace on save.

Your tokens seem to be in a random order, but the order is important, because the tokens are matched by Prism in that order. For example operator and punctuation should be last. Operators that are matched too early can prevent later matches. For example the minus sign in @load-sigs is matched as an operator:

I don't know Bro, but I assume that this 'italic':  /\b(TODO|FIXME|XXX)\b/, should be inside of a comment?

'comment': {
   pattern: /(^|[^\\$])#.*/,
   lookbehind: true,
   inside: {
       'italic':  /\b(TODO|FIXME|XXX)\b/,
   }
},

What minimizer do you use? If I use gulp to generate prism-bro.min.js I get a different result than yours. Maybe you used another version of gulp...

When you are finished you should also squash all your commits down into one or two nice commits, so that we don't have all the corrections in the log.

OK, I fixed the whitespaces, reordered the tokens, fixed the italics, and used Gulp for magnification instead of the other tool.

Thanks for your contribution!
It would be really nice if you could create a few test cases for the Bro language. The tests are in the directory tests/languages. You can run the tests with the command npm test. The syntax should be straight forward.

I missed that this was a PR for the master branch. You should create your PR for the gh-pages branch. I moved your commits there and removed them from master.

Guidelines regarding how to write tests can be found here: http://prismjs.com/test-suite.html

OK, and use the gh-pages branch for the test suites?

Yes. All PRs should be done on gh-pages ;)


Fix Creating a compiler plugin to export OpenAPI files of services in a package

Purpose

Fix #709

Automation tests

Unit tests - add

Code coverage information

Integration tests -add

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes/no
Ran FindSecurityBugs plugin and verified report? yes/no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes/no

Samples

Provide high-level details about the samples related to this feature

Related PRs

List any other related PRs

Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

Merge after merging this PR https://github.com/ballerina-platform/ballerina-lang/pull/34113


wrong package name for config controller

Package for config controller is named rbac (https://github.com/clastix/capsule/blob/master/controllers/config/manager.go#L4) instead of config

Nice catch, totally missed this.


keda controller enable verbose logging and debugging

How do I enable additional debugging and/or build version of keda controller I can debug?
I am gettign endless stream messages in keda operator logs (every second ...):
time="2019-06-06T16:52:02Z" level=info msg="Watching ScaledObject: default/keda-scaled-alek5"
time="2019-06-06T16:52:02Z" level=info msg="Created HPA with namespace default and name keda-hpa-kafka-source-alek1-qmzn7"
time="2019-06-06T16:52:02Z" level=error msg="Error updating scaledObject (default/keda-scaled-alek5) status: Operation cannot be fulfilled on scaledobjects.keda.k8s.io \"keda-scaled-alek5\": the object has been modified; please apply your changes to the latest version and try again"
time="2019-06-06T16:52:03Z" level=error msg="Error updating scaledObject (default/keda-scaled-alek5) status: Operation cannot be fulfilled on scaledobjects.keda.k8s.io \"keda-scaled-alek5\": the object has been modified; please apply your changes to the latest version and try again"
...

(base) aslom@m keda-testing $ k get pods |grep kafka-source-alek1-qmzn7
kafka-source-alek1-qmzn7-545965cd74-29fsq   1/1     Running   0          13h
(base) aslom@m keda-testing $ k apply -f keda-scaled-object-kafka-source-alek5.yaml
scaledobject.keda.k8s.io/keda-scaled-alek5 configured
(base) aslom@m keda-testing $ cat keda-scaled-object-kafka-source-alek5.yaml
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: keda-scaled-alek5
  namespace: default
  labels:
    deploymentName: kafka-source-alek1-qmzn7
spec:
  scaleTargetRef:
    deploymentName: kafka-source-alek1-qmzn7
  pollingInterval: 1  # Optional. Default: 30 seconds
  triggers:
  - type: kafka
    metadata:
      brokerList: my-cluster-kafka-brokers.kafka:9092
      consumerGroup: groupa4
      topic: alek1

Thank yo for any pointer/help in advance.

You can define it if you're using the helm chart, docs are being added here: https://github.com/kedacore/keda/pull/222
Will improve the description for them a bit so that it's clear what defines what

Hi, can you please try reinstalling KEDA using helm and report back if you encounter these errors?

Log levels setting documented here:
https://github.com/kedacore/keda#setting-log-levels

@zroubalik - the link doesn't seem to work. Can you share an updated link?

https://github.com/kedacore/keda/blob/main/BUILD.md#setting-log-levels


isVue3 is not defined

Hello,
I have a following issue. Looks to be a trivial one:
📦  Installing vue-cli-plugin-nativescript-vue...

+<EMAIL_ADDRESS>updated 1 package and audited 1300 packages in 10.86s

54 packages are looking for funding
  run `npm fund` for details

found 1 high severity vulnerability
  run `npm audit fix` to fix them, or `npm audit` for details
✔  Successfully installed plugin: vue-cli-plugin-nativescript-vue

? Enter a unique application identifier: org.nativescript.application
? Use HTML5 history mode? (Default: hash mode) No
? Is this a brand new project? (Default: Yes) Yes
? Dual Native AND Web development experience or a Native only? (Default: Dual) Dual Native AND Web
? What type of template do you want to start with? (Default: Simple) Nativescript-Vue-Web - The Simple template, but with NS-Vue like syntax for web components

🚀  Invoking generator for vue-cli-plugin-nativescript-vue...
adding to package.json
deleting from package.json
doing template rendering
 ERROR  ReferenceError: ejs:1
 >> 1| <%_ if (isVue3) { _%>
    2| import { createApp } from 'vue'
    3| import App from '~/App.vue';
    4| import { Page, ActionBar, GridLayout, Button, Img, Label } from 'nativescript-vue-web';

isVue3 is not defined
ReferenceError: ejs:1
 >> 1| <%_ if (isVue3) { _%>
    2| import { createApp } from 'vue'
    3| import App from '~/App.vue';
    4| import { Page, ActionBar, GridLayout, Button, Img, Label } from 'nativescript-vue-web';

isVue3 is not defined
    at eval (eval at compile (/usr/lib64/node_modules/@vue/cli/node_modules/ejs/lib/ejs.js:652:12), <anonymous>:10:8)
    at anonymous (/usr/lib64/node_modules/@vue/cli/node_modules/ejs/lib/ejs.js:682:17)
    at Object.exports.render (/usr/lib64/node_modules/@vue/cli/node_modules/ejs/lib/ejs.js:419:37)
    at renderFile (/usr/lib64/node_modules/@vue/cli/lib/GeneratorAPI.js:523:14)
    at /usr/lib64/node_modules/@vue/cli/lib/GeneratorAPI.js:312:27
    at Generator.resolveFiles (/usr/lib64/node_modules/@vue/cli/lib/Generator.js:268:13)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Generator.generate (/usr/lib64/node_modules/@vue/cli/lib/Generator.js:175:5)
    at async runGenerator (/usr/lib64/node_modules/@vue/cli/lib/invoke.js:111:3)
    at async invoke (/usr/lib64/node_modules/@vue/cli/lib/invoke.js:92:3)

I have the same issue when creating a new nativescript & web shared code application

I have the same issue. Any resolution on this?

Same issue here! I think it has to do with the @vue/cli version, but downgrading it doesn't seem to work either.

Same problem - any direction would be appreciated

Same issue here. I also tried downgrading @vue/cli without success.

This is an issue with with 'vue-cli-plugin-nativescript-vue' not being compatible with the newer @vue/cli versions. The plugin extends templates in the cli to scaffold a new app when you add the plugin. However it doesn't include the new 'isVue3' render option that was introduced to the @vue/cli's templates recently.
I have just created a pull request to fix this (which you can view here: https://github.com/nativescript-vue/vue-cli-plugin-nativescript-vue/pull/53). In the mean time downgrading @vue/cli version has worked for me. Downgrading to far seems to cause other issues but I have got it to work with cli version 4.0.0


Provide eq

Maybe @autodict https://github.com/smarie/python-autoclass/issues/7 will be a preferred way to go here. If one needs eq to be generated and be based on some attributes equalities, simply adding @autodict should work. Indeed this adds Mapping as a subclass, which implements it already by comparing the dicts


Fix CI: Pass certs to server

Signed-off-by: Vineet Badrinath<EMAIL_ADDRESS>

/approve
/lgtm


sidebar active items on custom domains

For some reason the active link in the sidebar isn't correct server-side. On client navigation it seems to be correct.

fixed in https://github.com/invertase/docs.page/pull/131


Too many false positives from WAF?

Hi, How is this even possible that it raises so many false flags?
For reference, website is behind cloudflare and it's a normal Wordpress website selfhosted with Docker, and I am using NPM with your integration
Thank you very much for your help in advance

I just noticed Open AppSec is not what I need since the behaviour analysis is too heavy.
Don't get me wrong. With NPM Vanilla my Host doesn't go down under DDoS attacks even with 50M PPS

Hi @NazgulCoder,
open-appsec, unlike traditional WAF is using Machine Learning as the security engine. Meaning the model needs to learn when first deployed and could raise some false positive, this is why we recommend starting in prevent mode and only after some time moving to prevent - you can read more here - https://docs.openappsec.io/how-to/track-learning-and-move-from-learn-detect-to-prevent
Regarding DDoS, you can set up rate limiting to protect your application from DDoS read more here -   https://docs.openappsec.io/how-to/setup-rate-limit-ddos-control

Hi @NazgulCoder,
open-appsec, unlike traditional WAF is using Machine Learning as the security engine. Meaning the model needs to learn when first deployed and could raise some false positive, this is why we recommend starting in prevent mode and only after some time moving to prevent - you can read more here - https://docs.openappsec.io/how-to/track-learning-and-move-from-learn-detect-to-prevent
Regarding DDoS, you can set up rate limiting to protect your application from DDoS read more here - https://docs.openappsec.io/how-to/setup-rate-limit-ddos-control

Hey thanks for the reply.
Regarding the WAF, I gave it a try again but even after several days (i get around 100 visits daily for each site, for a total of 2k visits per day) it still was getting many false flags.
Regarding the DDoS, I tried also your rate limit but I'm pretty sure the openappsec module implementation is just "too heavy" that drains too many resources.
Like I said, same DDoS attack with NPM and NPM + OpenAppSec have totally different behaviour and resources usage


Generate self signed certs for webhook

Signed-off-by: Matthew DeVenny<EMAIL_ADDRESS>Closes #2
K8s updates remove the ability to sign server certificates - so this update uses crypto/tls to generate self signed certificates for the web hook

lgtm


The theme doesn't change

Description:
The theme doesn't change
Steps to reproduce:

Start Messenger
Log In
Go to Theme and select Mosaic
Nothing changes

App version:
2.0.1-beta, downloaded ten minutes ago
OS:
Windows 10 Pro x64
Works in WhatsApp Web:
Extra:
No, nothing more as of now, but I'll make sure to suggest if I think of something :)

Here's the original poster of that bug.
The Messenger seems to have changed the Theme only after restarting the client, unlike in previous versions where it changed instantly.


Nested cat-datepicker inside cat-dropdown leads to misbehavior

The following problems occur when you open a dropdown in which a cat file picker is nested.

Issue without noAutoClose in cat-dropdown:

If you choose a date inside the datepicker it will also close the dropdown

Issue with notAutoClose in cat-dropdown:

If you select a date and the focus is still on the dropdown and you want to close the dropdown manually, the modal for selecting a date opens again.

If the first input field in the dropdown is a DatePicker, the focus is placed directly on the input field and the DatePicker opens automatically.

I create a branch for this ticket and in the angular project I add a simple example, where you can the the missbehvaior


Automatically choose docker compose run method

If the standalone "docker-compose" command exists, use that. Else, use "docker compose".
This should address https://github.com/getumbrel/llama-gpt/issues/62

It makes no sense, Compose V1 (docker-compose) is officially EoL and should not be supported.

Compose V1 support will no longer be provided after June 2023 and will be removed from all future Docker Desktop versions. If you’re still on Compose V1, we recommend you switch as soon as possible to leave time to address any issues with running your Compose applications.

See https://www.docker.com/blog/new-docker-compose-v2-and-v1-deprecation/ and https://github.com/docker/compose/tree/v1.


Integrate with InstallrApp

I suggested it in UserVoice: https://bitrise.uservoice.com/forums/235233-general/suggestions/12886044-add-support-for-uploading-artifacts-to-installr
InstallrApp https://www.installrapp.com/ is a service for distributing iOS and Android builds (similar to HockeyApp and others). They have an API http://help.installrapp.com/api/ and it would be great to be able to push .ipa's and .apk's there as well :)

Thanks for the request! I maked it for "idea/request for others", in case someone else would want to work on it before we could get to it ;)

This is now completed, thank @Redth !


Allow users to choose between the stable and git master versions of Codespell

Looks like it should be possible by passing args into the Dockerfile
https://help.github.com/en/actions/creating-actions/creating-a-docker-container-action#creating-an-action-metadata-file
And here:
https://docs.docker.com/engine/reference/builder/

Is this task complete?

Is this task complete?

I don't believe so, we still just load them in via requirements.txt which specifies a tagged version currently:
https://github.com/codespell-project/actions-codespell/blob/master/requirements.txt

uses: codespell-project/actions-codespell@v1
uses: codespell-project/actions-codespell@stable
uses: codespell-project/actions-codespell@master


bada-bib: 0.6.2 -> 0.7.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for bada-bib is: "A simple BibTeX Viewer and Editor"
meta.homepage for bada-bib is: "https://github.com/RogerCrocker/BadaBib"
meta.changelog for bada-bib is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.7.2 with grep in /nix/store/5i15798lyqxn49spryydaa67jmrzpq8s-bada-bib-0.7.2
found 0.7.2 in filename of file in /nix/store/5i15798lyqxn49spryydaa67jmrzpq8s-bada-bib-0.7.2

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
bada-bib

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5i15798lyqxn49spryydaa67jmrzpq8s-bada-bib-0.7.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A bada-bib https://github.com/r-ryantm/nixpkgs/archive/4975868d54636aae086d8e7cbf75c3723ce31db5.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5i15798lyqxn49spryydaa67jmrzpq8s-bada-bib-0.7.2
ls -la /nix/store/5i15798lyqxn49spryydaa67jmrzpq8s-bada-bib-0.7.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

bada-bib

Maintainer pings
cc @Cogitri for testing.

Doesn't seem to work:

Should be fixed now I think.

Result of nixpkgs-review pr 180296 run on x86_64-linux 1

1 package built:

bada-bib

Works :)


Fixes channels listed on the whois message to be links

There's probably a better way to do this...
 vs
Notice the commas

Alright, so if I'm correct, it's failing because of the comma before the channel name, like ,#channel.
For this fix, I'd rather have .join(", ") to keep the comma and the clickable channel names.
Then there is also the problem of +#channel and @#channel. Not sure how to address this (we can't make #something match globally or URLs will be pretty messed up) nor if we should address it as part of this PR. It depends if the solution to this problem affects or is affected by your fix or not.
Also, would you mind fixing the build? :-P

Not sure why the build is failing and have no experience with Jenkins.

In my opinion, the message parser should be fixed to support those instead of trying to rewrite the whois response to work with it. I have a feeling that kind of issue is bound to happen again.

I agree with @maxpoulin64. Any suggestion for this? It would be sad to refuse a PR that solves an actual issue in favor or a potential better fix that could not happen anytime soon.
@MaxLeiter, if you open Travis CI (not Jenkins :P), it will tell you that JS style has 2 issues:
if(key == "on") {
// ... should be ...
if (key === "on") {

Notice the whitespace after if and the triple-equal sign.
Also, you can run npm install && npm test locally and you'll have the same output, no need to parse Travis CI's output :-)

As per the IRC RFC, channel names can not contain commas, and thus that character can be used to as whitespace in inline parser. I will add this in #128.

Updated, but still has the issue with OP/voice, as @xPaw mentioned

I think there is no reason to keep this PR open as mine fixes the same issue.

@MaxLeiter, @maxpoulin64, @xPaw, now that #128 is merged, is this PR still relevant?
If the links are fixed, we could at least add a space between each comma to present it nicer, no?

I'll check out #128 when I get the chance, and there is a space after the commas

Closing in favour of #161.


arm: irq: Disable clang warning on VFP clobbering

Clang warns the user whenever an ISR invokes a function that is not marked with the interrupt attribute. This applies to the whole call chain, which makes it unfeasible to add the attribute to add it to all functions.
Instead, disable the clang warning just like it was done for the equivalent GCC one in 5b3f80094ec561758bd506c20c453703b859b8c6.
See also:
https://reviews.llvm.org/D28820

Maybe @stephanosio can clarify what his commit message meant.

https://github.com/zephyrproject-rtos/zephyr/issues/49631#issuecomment-1230352096


pip install failed

Howdy,
I am trying to install gloria via pip, and I did pip install git@github.com:marshuang80/gloria.git. I got this error:
ERROR: Invalid requirement: 'git@github.com:marshuang80/gloria.git' Hint: It looks like a path. File 'git@github.com:marshuang80/gloria.git' does not exist.
Could you help to take a look at it?
Many thanks!

Thank you, but I found another error when importing gloria. AttributeError: module 'gloria' has no attribute 'constants'

Or maybe you can directly download gloria package repository manually and place it where other package folders are. I think it would work since the folder gonna have init.py file there. They are mainly in site-packages folder or maybe in some other folder. But you need to find the folder on your system and place it there.

maybe we need to relocate constants or reimport constants in init.py file and use alias.

In gloria/gloria/init.py file try to add the following line
from . import constants
If you see the code they imported builder but not constants from the directory, maybe they didn't placed at the file at right place to import it along with other files. Try this

Thank you for the suggestion, I will try those in a couple of days, and I will inform you whether they fix this issue or not. Many thanks again!


fix namespace check when joining tables

The unit test is reproducing the issue. Even if the query is correct, a
"table not found error" was returned.
This PR is based on https://github.com/stephencelis/SQLite.swift/pull/916
Fix https://github.com/stephencelis/SQLite.swift/issues/777

Should be fixed by #811. Tell me if there is any problem with that.


Istio Service Mesh Extended for Event Driven Messaging

To reduce the moving parts and complexity in my microservices setup, instead of using Kafka, am thinking if Istio Service Mesh can be extended for Event Driven Messaging?
Nathan Aw (Singapore)
SOURCE: https://stackoverflow.com/questions/62163619/can-i-use-my-service-mesh-for-event-driven-messaging/62168409#62168409

Hey @nathanawmk , can you expand more on what this would look like? It sounds like you want Istio to replace Kafka which doesn't seem in the scope of the project but I may he misunderstanding

Its not clear what the request is, and seems out of scope. Based on this, I will close


add Texas to list of regions

I have added a geojson boundary for Texas along with an entry in config/regions.yaml. I have not added the osmcha.id as directed in CONTRIBUTING.md.

Hi, is there anything I can do to expedite this process? I can make an OSMCha filter for the region if that would help speed things up :smiley:

Thank you!

🚀 Deployed: https://welcome.osm.be/north-america/texas/list
List will be automatically updated tomorrow!


RocketMQLocalTransactionListener checkLocalTransaction  number of times

I hope to get the data of the number of times the transaction is backchecked when the transaction is backchecked, so as to facilitate the subsequent judgment. For example, when the transaction is backchecked more than three times, the transaction is rolled back and logged.

I found a solution
i use rocketmq-spring-boot-starter.version  2.1.0
when implement  RocketMQLocalTransactionListener checkLocalTransaction(Message msg) method
String checkTimes = (String) msg.getHeaders().get(MessageConst.PROPERTY_TRANSACTION_CHECK_TIMES);
if use rocketmq-client   I think it's easier


⚠ pan-serv00 has degraded performance

In 63bc6d6, pan-serv00 (https://cloud.010702.xyz/) experienced degraded performance:

HTTP code: 403
Response time: 19327 ms

Resolved: 毒蛇影院 performance has improved in c0c4452 after 82 days, 22 hours, 58 minutes.


Closing Component channels that are not explicitly marked as ports

Somewhere under the hood, it seems as though goflow is calling close on any node in its graph if that node has a Out channel field, even if that port is not connected in the graph. For example,
package main

import (
	"log"

	"github.com/trustmaster/goflow"
)

type Receiver struct {
	name string
	In   <-chan []byte
	Out  chan<- []byte
}

func NewReceiver(name string) *Receiver {
	return &Receiver{name: name}
}

func (db *Receiver) Process() {
	for in := range db.In {
		log.Printf("Processing Receiver for node %s", db.name)
		db.Out <- in
	}
}

type DataBuffer struct {
	name string
	In   <-chan []byte
	Out  chan<- []byte
}

func NewDataBuffer(name string) *DataBuffer {
	return &DataBuffer{name: name}
}

func (db *DataBuffer) Process() {
	for in := range db.In {
		log.Printf("Processing for node %s: %v", db.name, in)
	}
}

func main() {
	g := goflow.NewGraph()

	g.Add("greeter", new(Receiver))
	g.Add("printer", new(DataBuffer))

	g.Connect("greeter", "Out", "printer", "In")

	g.MapInPort("In", "greeter", "In")

	in := make(chan []byte)

	g.SetInPort("In", in)

	wait := goflow.Run(g)

	in <- []byte{1, 0}

	close(in)

	<-wait
}

outputs
2009/11/10 23:00:00 Processing Receiver for node
2009/11/10 23:00:00 Processing for node : [1 0]
panic: close of nil channel

~Changing the name of DataBuffer.Out to DataBuffer.LeaveMeAlone~ Making the field private (or removing it)  fixes the issue.
I'm new to goflow (looks neat!) so maybe this is intended behaviour. From a newbie perspective it is unexpected.

This is intended behavior. If you want to hide a channel from GoFlow's graph, make it private.

@trustmaster  correct me please if i'm wrong:
goflow executor will close all channel ports that would not have their connections before starting the graph, so we should not care about closing such channels ourselves?

@Altenrion GoFlow only takes care of closing output ports that have channels attached to them when component is finished (not when started). If outputs are not attached to anything, it will not close them.
If your question is about inputs, e.g. what if you have multiple of them like
type Receiver struct {
	name string
	In   <-chan []byte
	Opt <-chan int
	Out  chan<- []byte
}

If e.g. Opt is not connected to anything, GoFlow runtime doesn't do anything about it. What matters is how your Process() handles such optional ports. An example would be

func (db *Receiver) Process() {
	// Default value
	opt := 0
	// Check if Opt is connected
	if db.Opt != nil {
		opt = <-db.Opt
	}
	for in := range db.In {
		log.Printf("Processing Receiver for node %s", db.name)
		// Do something with opt here
		db.Out <- in
	}
}

@trustmaster thanks for examples.
I found that GoFlow closes output ports (attached with other side component port) when processing of component ends.
In my case i have "infinite" flows that starts & runs for a long period sequentially.
So for handling "optional" channel ports i made a higher management wrapper, that wraps the component, and checks if in the flow scenario this exact port is not intended to be connected, than fire "portCloser(portname)" logic on each component, for ports that are marked as non attached.
This was a non trivial endeavour but what i came up with is like that:
`
type PortCloser interface {
ClosePort(name string)
}
type Receiver struct {
name string
In   <-chan []byte
Opt <-chan int
Out  chan<- []byte
}
func (r *Receiver) ClosePort(name string) {
switch name {
case "Opt":
ch = make(chan int)
close(ch)
r.Opt = ch
}
}
func (r *Receiver) Process() {
.....
}
`
Right now i'm thinking if this is optimal solution or not, and will be testing different cases inside Process() around this.
Will definitely try your recommendation about db.Opt != nil.

@Altenrion I'm not sure I understand what you are trying to achieve. If you explain what your long-running component does and why you need to close the ports when running it would be easier for me.
In FBP, the "close all outputs on end" is a typical tactic for graceful shutdown of the graph. It does not work in 100% of cases though, and it is not intended for anything other than just graceful shutdown.


Missing parent: toll-like receptor signaling pathway

Make an is_a relationship between
toll-like receptor signaling pathway GO:0002224 and cell surface receptor signaling pathway GO:0007166

There are intracellular Toll receptors, see https://www.frontiersin.org/articles/10.3389/fimmu.2014.00461/full

TLRs localize to the cell surface or to intracellular compartments such as the ER, endosome, lysosome, or endolysosome, and they recognize distinct or overlapping PAMPs such as lipid, lipoprotein, protein, and nucleic acid


a few formatting fixes

There was an error while validating job 5628f087-4f67-4ac2-a99d-bcfe718cfe75 Message:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
Details:Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase+ContextCommandException:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
---> System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Error clonning repository ---> LibGit2Sharp.NotFoundException: Branch 300 not found. Available branches: origin/master
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetBranch(Repository repo, String name)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass30.<>c__DisplayClass34.b__29(KeyValuePair2 branch)    at System.Collections.Generic.List1.ForEach(Action1 action)    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass30.<CloneRepository>b__28()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func1 func)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)    --- End of inner exception stack trace ---    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository()
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetVersionAsync(String path)
at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)    --- End of inner exception stack trace ---    --- End of inner exception stack trace ---    at Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase.ThrowIfErrorsLogged()    at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)
at Microsoft.LightweightJobEngine.Core.Language.JobWorkflow1.RunCommands(Guid jobId, IDictionary2 context, CancellationTokenSource cancelSignal)


[DOC] RedisJSON Spack installation

Description of the question
The RedisJSON spack installation should be documented here: https://deephyper.readthedocs.io/en/develop/install/spack.html
and referenced here: https://deephyper.readthedocs.io/en/develop/install/pip.html#distributed-computation

base installation on Redis website can be as simple:
wget https://download.redis.io/redis-stable.tar.gz
tar -xzvf redis-stable.tar.gz
cd redis-stable
make
make PREFIX=/some/other/directory install

published at: https://deephyper.readthedocs.io/en/latest/install/spack.html


feat: add ceph-mon service to osds hosts firewall

Hosts that carry just OSDs appear to require ceph-mon service firewalls to ensure communication for Ceph MDS connectivity.

nit: Should probably have a release note

/cherry-pick stackhpc/2023.1

Shouldn't you merge first?

/cherry-pick stackhpc/2023.1

Shouldn't you merge first?

If the PR hasn't merged, it'll cherry-pick it once it's merged

/cherry-pick stackhpc/2023.1

Shouldn't you merge first?

The cherry pick will only happen after this has merged. https://github.com/googleapis/repo-automation-bots/tree/main/packages/cherry-pick-bot#on-unmerged-pull-request
This way I can line up the cherry-pick now and not after it has merged.

This is odd behaviour, can you confirm that you see OSDs listening on ports 3300 or 6789?

It is also worth noting that the highest port number used by Ceph was increased in [1] but the firewalld service definition was not updated right away [2].
[1] https://github.com/ceph/ceph/pull/42210
[2] https://github.com/firewalld/firewalld/issues/1329

It is also worth noting that the highest port number used by Ceph was increased in [1] but the firewalld service definition was not updated right away [2].
[1] ceph/ceph#42210 [2] firewalld/firewalld#1329

I think maybe solution is a bit clouded and maybe the firewall change here is a redherring. I don't have any appetite to revert the change in the environment where the problem was observed.
It was certainly missing ceph service from the host and enabling the firewall without ceph-mon caused sssd within the slurm cluster to fail. Though things didn't properly return until after a reboot of NFS hosts.
I think this could be tested within virtualised environment to be sure if this is required at all. Will mark as draft for now.

This is odd behaviour, can you confirm that you see OSDs listening on ports 3300 or 6789?

Checked impacted hosts those ports are not in use.


[EventHubs] Fix CI API View failure (unapproved by design)

currently due to the issue in the APIView tool: https://github.com/Azure/azure-sdk-tools/issues/1880, the API view of EventHubs is not generated correctly.
We have decided not to approve the wrong API view, letting the CI fail everyday as a reminder to fix the issue.
However, this might not be a good approach as we tend to ignore other CI errors (e.g. https://github.com/Azure/azure-sdk-for-python/issues/20565).
we should keep this in mind, do our daily check and fix the API view ASAP.

error: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=1097421&view=logs&j=b70e5e73-bbb6-5567-0939-8415943fadb9&t=d31d5e2d-d8df-53bc-da94-87e19f5b2ee4&l=25

get approval on this api view: https://apiview.dev/Assemblies/Review/3e49ce1ca870470289360023dcb51edf?diffRevisionId=4379fb2d3d214ceb99a7a62bfda6befe&doc=False&diffOnly=False


clean up tasks fails with wrong number of arguments (given 0, expected 1)

Describe the bug
The clean up tasks tasks fails every time.
To Reproduce
Steps to reproduce the behavior (example):

Deploy a bosh director 270.12.0
Wait until the first scheduled task clean up runs

Expected behavior
Don't error but clean up tasks.
Logs
E, [2020-03-22T00:00:02.054392 #28825] [task:104624] ERROR -- DirectorJobRunner: wrong number of arguments (given 0, expected 1)
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/jobs/scheduled_tasks_cleanup.rb:18:in `initialize'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/job_runner.rb:84:in `new'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/job_runner.rb:84:in `perform_job'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/job_runner.rb:34:in `block in run'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh_common-0.0.0/lib/common/thread_formatter.rb:50:in `with_thread_name'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/job_runner.rb:34:in `run'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/jobs/base_job.rb:9:in `perform'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/jobs/db_job.rb:32:in `block in perform'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/bosh-director-0.0.0/lib/bosh/director/jobs/db_job.rb:98:in `block (3 levels) in run'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:1077:in `block in spawn_threadpool'
/var/vcap/data/packages/director/4de68307cefbbb6cbb7e4a39d40adef9bf335e6a/gem_home/ruby/2.6.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Versions (please complete the following information):

Infrastructure: any
270.12.0
ubuntu-xenial/621.59

Deployment info:
Defaults are used for director.tasks_cleanup_schedule. max_tasks=10000.

strange, the _ should mean the params are ignored https://github.com/cloudfoundry/bosh/blob/master/src/bosh-director/lib/bosh/director/jobs/scheduled_tasks_cleanup.rb#L18
looks like we could either default to {} https://github.com/cloudfoundry/bosh/blob/master/src/bosh-director/lib/bosh/director/jobs/scheduled_orphaned_network_cleanup.rb#L24  or remove completely https://github.com/cloudfoundry/bosh/blob/master/src/bosh-director/lib/bosh/director/jobs/scheduled_orphaned_vm_cleanup.rb#L14


Re organise roles structure

We now use numbers as prefix for tasks files. It is easier and more
visual to determine the execution order.
Signed-off-by: Sébastien Han<EMAIL_ADDRESS>

To much work, few benefit, only cosmetic...


What is OPT_NONEG doing?

And what are the flags for argparse_init? The example only shows 0.

hi,
OPT_BOOLEAN and OPT_BIT types support negation, you can use no- prefix to negate the value, for example:
OPT_BIT(0, "read", &perms, "read perm", NULL, PERM_READ, OPT_NONEG),
OPT_BIT(0, "write", &perms, "write perm", NULL, PERM_WRITE),
OPT_BIT(0, "exec", &perms, "exec perm", NULL, PERM_EXEC),

--write turns on write permission, --no-exec means no exec permission.
But, if you don't want this feature, you can use OPT_NONEG flag to turn off this feature. In the example above, --no-write and --no-exec is valid, but --no-read is invalid.

Can those flags be put into argparse_init(&argparse, options, usage, 0);. See the 0 at the end?

@CMCDragonkai
NO, OPT_NONEG is per OPT flag.


Temporal Fields: 'NoneType' object has no attribute 'upper'

Superset version
0.17.1
Expected results
Temporal fields with None as their Type should either function the same as   as their Type or not be created at all.
Actual results
Slice fails to load with error 'NoneType' object has no attribute 'upper'
Trace:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/superset/viz.py", line 265, in get_payload
    df = self.get_df()
  File "/usr/local/lib/python2.7/site-packages/superset/viz.py", line 109, in get_df
    self.results = self.datasource.query(query_obj)
  File "/usr/local/lib/python2.7/site-packages/superset/connectors/sqla/models.py", line 525, in query
    sql = self.get_query_str(engine, qry_start_dttm, **query_obj)
  File "/usr/local/lib/python2.7/site-packages/superset/connectors/sqla/models.py", line 429, in get_query_str
    time_filter = dttm_col.get_time_filter(from_dttm, to_dttm)
  File "/usr/local/lib/python2.7/site-packages/superset/connectors/sqla/models.py", line 68, in get_time_filter
    col >= text(self.dttm_sql_literal(start_dttm)),
  File "/usr/local/lib/python2.7/site-packages/superset/connectors/sqla/models.py", line 117, in dttm_sql_literal
    self.type, dttm)
  File "/usr/local/lib/python2.7/site-packages/superset/db_engine_specs.py", line 239, in convert_dttm
    if target_type.upper() in ('DATETIME', 'DATE'):
AttributeError: 'NoneType' object has no attribute `'upper'

Steps to reproduce
Create an additional field on a table and mark it as temporal. The Type will default to None rather than   as per previous versions. This cases a slice using this as the Time Column to fail with the above error.
Two possible ways to fix: Either default back to   (blank)  rather than None or NULL, or add in some handling for None values in convert_dttm.

Fixed in 0.17.3

Notice: this issue has been closed because it has been inactive for 382 days. Feel free to comment and request for this issue to be reopened.


Align Multiple inputs horizontally

Currently all field inputs are vertically alligned. Even if I add schema.fieldClasses: 'col-md-4' and schema. styleClasses: 'col-md-3', width of field input doesn't change.
I found that css class .vue-form-generator .form-group has width: 100% which is not effected by schema. styleClasses and schema.fieldClasses.
Ofcourse field should be of width 100% if style classes are not provided.
Currently I am using work-around for this by removing "width:100%" from class vue-form-generator .form-group and adding rule form-group:not([class*=" col-"]){width:100%}. It could be helpful for horizontal inputs.

and what is the option for removing VFG's CSS since all we're doing is installing it via NPM and adding
import VueFormGenerator from "vue-form-generator";
Vue.use(VueFormGenerator);

@vesper8 you should have something like this
// the "full" way
<script>
  import VueFormGenerator from "vue-form-generator";
  import "vue-form-generator/dist/vfg.css";  // optional full css additions
</script>

// the "core" way
<script>
  import VueFormGenerator from "vue-form-generator/dist/vfg-core.js";
  import "vue-form-generator/dist/vfg-core.css";  // optional core css additions
</script>

You would just omit the '.css' import.


nvidia dali support

🚀 Feature
Re-open the case for #513. i.e. Support nvidia dali iterator as a possible data loader.
Pitch
I have submitted a wip pull request to the master. #789
I would like to know if it is ok for the test to be depending on nvidia-dali in addition to apex.

The dali iterator does not support resetting while epoch is not finished. I suppose I shall provide a warning for that.

@smallzzy this is sick. super excited by this feature

@smallzzy we have cleared the API and now it should be stable... Mind resume your addition?

Some news about DALI support?

@brunoalano interested in implementing DALI?

@Borda Do you have a pipeline what should have be done to support it? But I'm available to do that with a minimal guidance (started using PyTorch Lightning and DALI these days)

we had almost done PR some time ago so I use you can resume from that point... #789

Is anyone working on this currently?

@irustandi I think no one

I tried converting PyTorch Lightning's MNIST example to use DALI, it seems to work out of the box without requiring modification to the PyTorch Lightning's internals. See below.
https://gist.github.com/irustandi/3d180ed3ec9d7ff4e73d3fdbd67df3ca

@irustandi cool, mind add it as example, send a PR?

@Borda sure, will create the PR.

I want to know that if dali with ddp and amp and dali's pipeline in distributed situation can work well? And have a perfect performance?

Good question, I believe so... @awaelchli?


feat: try to canonicalize safe and finalized

this is an initial attempt to address #2964
this needs some additional checks, I think we need to do additional block height checks
wdyt @Rjected

Codecov Report

Merging #2971 (4925084) into main (2e406c1) will decrease coverage by 35.79%.
The diff coverage is 0.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@             Coverage Diff             @@
##             main    #2971       +/-   ##
===========================================
- Coverage   70.96%   35.18%   -35.79%
===========================================
  Files         520      520
  Lines       67149    67175       +26
===========================================
- Hits        47654    23634    -24020
- Misses      19495    43541    +24046

Flag
Coverage Δ

integration-tests
?

unit-tests
35.18% <0.00%> (-30.69%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

crates/consensus/beacon/src/engine/forkchoice.rs
36.61% <0.00%> (-28.77%)
:arrow_down:

crates/consensus/beacon/src/engine/mod.rs
46.86% <0.00%> (-36.48%)
:arrow_down:

... and 315 files with indirect coverage changes

@Rjected reactivating this, because I think this is still a valid change


fix: define remote user

See: #23

:tada: This PR is included in version 1.2.6 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Grid Generation Error

I am trying to generate a grid roadnet of 1x8 dimensions ( eight consecutive intersections)
However executing the grid generator with params 1 and 8 is returning a grid of shape 1x2.
Any though?

Emmm, I can't figure out why this happens. Could you please try to read the code of "generate_json_from_grid.py", which is a very simple and clear code.


Abstraction for adding listeners to observables of the paper land model

One of the more complicated parts of program code right now is creating "observer" program code that listens for changes to "observable" program code in another program. The "observer" and "observable" could be detected in any order and that needs to be handled well. Currently this requires complicated listeners in program code. This should be abstracted in board.
Brainstorming notes so far:

/**
 * @param componentName {string} - name of the model component you expect to be in the model
 * @param handleComponent {function} - function that will be called if the component exists and when it is added to the model
 */
paperLand.addModelObserver( componentName: string, handleComponent: ( component ) => {} )
    handleComponentExists():
        - call handleComponent( component )
        - Add a listener that will call handleComponentDoesNotExist() when component is removed (with self removal)
    handleComponentDoesNotExist():
        - Add a listener that will call handleComponentExists() when the component is added to the model (with self removal)

    - if model component exists, handleComponentExists()
    - if model component does not exist, handleComponentDoesNotExist()

paperLand.removeModelObserver( componentName: string, handleComponent: ( component ) => {} )
    - if model component exists
        call handleComponent( component )

    - Remove any listeners on the global model waiting for the component to be added/removed

Usage could look like this:
Usage might look like this:

// in onProgramAdded
scratchpad.altitudeListener = altitude => {
  console.log( altitude );
}
scratchpad.handleAltitudePropertyAdded = altitudeProperty => {
  altitudeProperty.link( scratchpad.altitudeListener );
}
scratchpad.handleAltitudePropertyRemoved = altitudeProperty => {
  altitudeProperty.unlink( scratchpad.altitudeListener );
}

phet.paperLand.addModelObserver( 'altitudeProperty', scratchpad.altitudeListener );

// then in onProgramRemoved
phet.paperLand.removeModelObserver( 'altitudeProperty', altitude.handleAltitudePropertyRemoved );
delete scratchpad.handleAltitudePropertyAdded;
delete scratchpad.handleAltitudePropertyRemoved;
delete scratchpad.altitudeListener;

One downside of this is that we need handleAltitudePropertyAdded and handleAltitudePropertyRemoved. These handle
the link and unlink. If that was done in the addModelObserver they wouldn't be necessary. But I wanted this
to support any type of model component (maybe it uses addListener instead of link) or maybe even a variable that isn't observable and doesn't change.

As part of this, we should change the global modelProperty to be a map. We have Emitters that let us know when something is added and removed now.

Seems to be working well! But needs a lot more testing. These could be disruptive changes and I didn't get to a commit point but here is progress:

Subject: [PATCH] Use Checkbox directly, remove superclass, see https://github.com/phetsims/gravity-force-lab-basics/issues/324"
---
Index: client/board/entry.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/client/board/entry.js b/client/board/entry.js
--- a/client/board/entry.js	(revision 6c2425f66b8c5bf9fcaf39f864d82106277bc4dc)
+++ b/client/board/entry.js	(date<PHONE_NUMBER>281)
@@ -10,9 +10,9 @@
 import React from 'react';
 import ReactDOM from 'react-dom';
 import styles from './BoardMain.css';
-import paperLand from './paperLand.js';
 import PaperLandControls from './PaperLandControls.js';
 import SceneryDisplay from './SceneryDisplay.js';
+require( './boardModel.js' );

 // constants
 const DISPLAY_SIZE = new phet.dot.Dimension2( 640, 480 );
@@ -145,63 +145,64 @@
   console.warn( 'createAndLoadWrappedAudioBuffer not defined' );
 }

-// Emits events when model components are added or removed, to be used in program code. Emits with
-// {string} - name of the model component
-// {*} - Reference to the component being added or removed
-paperLand.modelComponentAddedEmitter = new window.phet.axon.Emitter();
-paperLand.modelComponentRemovedEmitter = new window.phet.axon.Emitter();
-
-/**
- * Adds a model component to the model Object with the provided name. Emits events so client code can observe
- * changes to the model.
- * @param {string} componentName
- * @param {*} componentObject - any model component (Property, or object with multiple Properties and values)
- */
-paperLand.addModelComponent = ( componentName, componentObject ) => {
-  const existingModel = modelProperty.value;
-  if ( existingModel[ componentName ] === undefined ) {
-
-    // Update the model Property, which is also our map for name -> component
-    modelProperty.value = {
-      [ componentName ]: componentObject,
-
-      // spread operator copies existing model into a new object
-      ...existingModel
-    };
-
-    paperLand.modelComponentAddedEmitter.emit( componentName, componentObject );
-  }
-  else {
-    console.warn( `Model already has component with name ${componentName}` );
-  }
-};
-
-/**
- * Remove a component with the provided name from the model. Updates the global modelProperty which is our map
- * of all model components and also emits a separate Emitter.
- * @param {string} componentName
- */
-paperLand.removeModelComponent = componentName => {
-  const existingModel = modelProperty.value;
-  const componentObject = existingModel[ componentName ];
-
-  if ( componentObject === undefined ) {
-    console.warn( `Model does not have component with name ${componentName}` );
-  }
-  else {
-
-    // delete the object from the global model and then reassign to trigger a Property change
-    const objectCopy = { ...existingModel };
-    delete objectCopy[ componentName ];
-    modelProperty.value = objectCopy;
+//
+// // Emits events when model components are added or removed, to be used in program code. Emits with
+// // {string} - name of the model component
+// // {*} - Reference to the component being added or removed
+// paperLand.modelComponentAddedEmitter = new window.phet.axon.Emitter();
+// paperLand.modelComponentRemovedEmitter = new window.phet.axon.Emitter();

-    // emit events, passing the componentObject through so that client can dispose of various objects
-    paperLand.modelComponentRemovedEmitter.emit( componentName, componentObject );
-
-    // dispose the component when we are done with it, if supported
-    componentObject.dispose && componentObject.dispose();
-  }
-};
+// /**
+//  * Adds a model component to the model Object with the provided name. Emits events so client code can observe
+//  * changes to the model.
+//  * @param {string} componentName
+//  * @param {*} componentObject - any model component (Property, or object with multiple Properties and values)
+//  */
+// paperLand.addModelComponent = ( componentName, componentObject ) => {
+//   const existingModel = modelProperty.value;
+//   if ( existingModel[ componentName ] === undefined ) {
+//
+//     // Update the model Property, which is also our map for name -> component
+//     modelProperty.value = {
+//       [ componentName ]: componentObject,
+//
+//       // spread operator copies existing model into a new object
+//       ...existingModel
+//     };
+//
+//     paperLand.modelComponentAddedEmitter.emit( componentName, componentObject );
+//   }
+//   else {
+//     console.warn( `Model already has component with name ${componentName}` );
+//   }
+// };
+//
+// /**
+//  * Remove a component with the provided name from the model. Updates the global modelProperty which is our map
+//  * of all model components and also emits a separate Emitter.
+//  * @param {string} componentName
+//  */
+// paperLand.removeModelComponent = componentName => {
+//   const existingModel = modelProperty.value;
+//   const componentObject = existingModel[ componentName ];
+//
+//   if ( componentObject === undefined ) {
+//     console.warn( `Model does not have component with name ${componentName}` );
+//   }
+//   else {
+//
+//     // delete the object from the global model and then reassign to trigger a Property change
+//     const objectCopy = { ...existingModel };
+//     delete objectCopy[ componentName ];
+//     modelProperty.value = objectCopy;
+//
+//     // emit events, passing the componentObject through so that client can dispose of various objects
+//     paperLand.modelComponentRemovedEmitter.emit( componentName, componentObject );
+//
+//     // dispose the component when we are done with it, if supported
+//     componentObject.dispose && componentObject.dispose();
+//   }
+// };

 // Update the sim design board based on changes to the paper programs.
 const updateBoard = presentPaperProgramInfo => {
Index: client/board/boardModel.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/client/board/boardModel.js b/client/board/boardModel.js
new file mode 100644
--- /dev/null	(date<PHONE_NUMBER>467)
+++ b/client/board/boardModel.js	(date<PHONE_NUMBER>467)
@@ -0,0 +1,164 @@
+import paperLand from './paperLand.js';
+
+// Map<string, Object> - keys are the name of the model component, values are any kind of model component
+const boardModel = new Map();
+
+const componentNameToAddedListenerMap = new Map();
+const componentNameToRemovedListenerMap = new Map();
+
+// Emits events when model components are added or removed, to be used in program code. Emits with two args
+// {string} - name of the model component
+// {*} - Reference to the component being added or removed
+paperLand.modelComponentAddedEmitter = new window.phet.axon.Emitter();
+paperLand.modelComponentRemovedEmitter = new window.phet.axon.Emitter();
+
+/**
+ * Adds a model component to the model Object with the provided name. Emits events so client code can observe
+ * changes to the model.
+ * @param {string} componentName
+ * @param {*} componentObject - any model component (Property, or object with multiple Properties and values)
+ */
+paperLand.addModelComponent = ( componentName, componentObject ) => {
+  if ( boardModel.get( componentName ) === undefined ) {
+    boardModel.set( componentName, componentObject );
+    paperLand.modelComponentAddedEmitter.emit( componentName, componentObject );
+  }
+  else {
+    console.warn( `Model already has component with name ${componentName}` );
+  }
+};
+
+/**
+ * Remove a component with the provided name from the model. Updates the global modelProperty which is our map
+ * of all model components and also emits a separate Emitter.
+ * @param {string} componentName
+ */
+paperLand.removeModelComponent = componentName => {
+  const componentObject = boardModel.get( componentName );
+
+  if ( componentObject === undefined ) {
+    console.warn( `Model does not have component with name ${componentName}` );
+  }
+  else {
+
+    // delete the object from the global model and then reassign to trigger a Property change
+    boardModel.delete( componentName );
+
+    // emit events, passing the componentObject through so that client can dispose of various objects
+    paperLand.modelComponentRemovedEmitter.emit( componentName, componentObject );
+
+    // dispose the component when we are done with it, if supported
+    componentObject.dispose && componentObject.dispose();
+  }
+};
+
+const addListenerToModelChangeEmitter = ( componentName, emitter, listener, listenerMap ) => {
+  emitter.addListener( listener );
+
+  if ( !listenerMap.has( componentName ) ) {
+    listenerMap.set( componentName, [] );
+  }
+  listenerMap.get( componentName ).push( listener );
+};
+
+const removeListenerFromModelChangeEmitter = ( componentName, emitter, listener, listenerMap ) => {
+  emitter.removeListener( listener );
+
+  if ( !listenerMap.has( componentName ) ) {
+    throw new Error( 'listenerMap does not have provided listener.' );
+  }
+  else {
+    const listeners = listenerMap.get( componentName );
+    const index = listeners.indexOf( listener );
+    if ( index > -1 ) {
+      listeners.splice( index, 1 );
+
+      if ( listeners.length === 0 ) {
+        listenerMap.delete( componentName );
+      }
+    }
+    else {
+      throw new Error( 'listener was not in the array for component' );
+    }
+  }
+};
+
+paperLand.addModelObserver = ( componentName, handleComponentAttach, handleComponentDetach ) => {
+
+  // Component exists in the model - do whatever work is needed on attach and add listeners to watch for when
+  // the component is removed again.
+  const handleComponentExists = component => {
+    handleComponentAttach( component );
+
+    const componentRemovedListener = removedComponentName => {
+      if ( componentName === removedComponentName ) {
+        handleComponentDoesNotExist( component );
+        removeListenerFromModelChangeEmitter( componentName, paperLand.modelComponentRemovedEmitter, componentRemovedListener, componentNameToRemovedListenerMap );
+      }
+    };
+    addListenerToModelChangeEmitter( componentName, paperLand.modelComponentRemovedEmitter, componentRemovedListener, componentNameToRemovedListenerMap );
+  };
+  const handleComponentDoesNotExist = component => {
+    if ( component !== undefined ) {
+      handleComponentDetach( component );
+    }
+
+    const componentAddedListener = ( addedComponentName, addedComponent ) => {
+      if ( componentName === addedComponentName ) {
+        handleComponentExists( addedComponent );
+        removeListenerFromModelChangeEmitter( componentName, paperLand.modelComponentAddedEmitter, componentAddedListener, componentNameToAddedListenerMap );
+      }
+    };
+    addListenerToModelChangeEmitter( componentName, paperLand.modelComponentAddedEmitter, componentAddedListener, componentNameToAddedListenerMap );
+  };
+
+  if ( boardModel.has( componentName ) ) {
+
+    // component already exists in the model, handle it and wait for removal
+    handleComponentExists( boardModel.get( componentName ) );
+  }
+  else {
+
+    // component does not exist yet, wait for it to be added
+    handleComponentDoesNotExist();
+  }
+};
+
+paperLand.removeModelObserver = ( componentName, handleComponentDetach ) => {
+  if ( boardModel.has( componentName ) ) {
+    handleComponentDetach( boardModel.get( componentName ) );
+  }
+
+  // remove any modelComponentAdded/modelComponentRemoved emitter listeners that were added on
+  // addModelObserver...
+  const componentAddedListeners = componentNameToAddedListenerMap.get( componentName );
+  if ( componentAddedListeners ) {
+    while ( componentAddedListeners.length > 0 ) {
+      paperLand.modelComponentAddedEmitter.removeListener( componentAddedListeners.pop() );
+    }
+  }
+
+  const componentRemovedListeners = componentNameToRemovedListenerMap.get( componentName );
+  if ( componentRemovedListeners ) {
+    while ( componentRemovedListeners.length > 0 ) {
+      paperLand.modelComponentRemovedEmitter.removeListener( componentRemovedListeners.pop() );
+    }
+  }
+};
+
+paperLand.addModelPropertyLink = ( componentName, listener ) => {
+  paperLand.addModelObserver(
+    componentName,
+    component => component.link( listener ),
+    component => component.unlink( listener )
+  );
+};
+
+paperLand.removeModelPropertyLink = ( componentName, listener ) => {
+  paperLand.removeModelObserver(
+    componentName,
+    component => component.unlink( listener )
+  );
+};
+
+export default boardModel;
\ No newline at end of file

This was pushed in the above commit. I tested out in basic programs and also added some unit tests for this functionality in particular. It seemed worthwhile because of the listener complexity needed here. "listener soup" is an appropriate description.
As part of this we changed modelProperty to be a Map. That impacts all existing program code, we need to update them.

Oops, this isn't working for adding mutliple model observers to the same observable. Not sure why yet, but the implementation should handle that.

While porting the IDRC programs I am finding it would be helpful to have similar functions for controllers too so you can modify a Property only when it exists in the model. It could be easy, Ill add it now.

That was added with unit tests for it. I am sure we will continue to iterate on this and create new abstractions as we identify pain points, but this is working well enough to close.


Fix call len call to get number of edges

Fix a call to len which should be using the method. I believe .edges used to work in previous networkx versions but this is more correct.
Fixes #36

Thank you! @Nyecarr


DFU Stuck on Connecting

We have a device that has DFU enabled on it, and my iOS code was working, but recently stopped working (around the time of iOS 10 and Swift 3). The latest issue that I have is that I am getting stuck on the didChangeStateTo call back of "connecting". I set the flag high on the DFU service so that I disconnect and change to the bootloader so I am getting DFUTarg and the advertisement. I set the DFUFirmware with a .bin and .dat file from my server. I connect to the DFUTarg peripheral, and once the connection is complete, the following code executes:
let initiator = DFUServiceInitiator(centralManager: self.centralManager!, target: peripheral)
initiator.withFirmwareFile(firmware)
// Optional:
// initiator.forceDfu = true/false; // default false
// initiator.packetReceiptNotificationParameter = N; // default is 12
initiator.logger = self // - to get log info
initiator.delegate = self // - to be informed about current state and errors
initiator.progressDelegate = self // - to show progress bar
// initiator.peripheralSelector = ... // the default selector is used
initiator.start()

I then get the call back response of "Connecting" and it gets stuck there. I seem to be implementing the code from nRFToolbox the same way, and as I said before, it seemed to be working fine before.
This is a pretty hot issue for my company, so any help would be greatly appreciated. Thanks a bunch.

Hi guys!
Did you already fixed this issue? I'm stucked on it too.
Thanks!

Did you try branch Version3? It will soon replace master.

Now I am getting a EXC_BAD_ACCESS(code=1,address=0x10) on initiator.with(firmware: firmware). I was not getting this issue before with the master branch. I am printing my details of the firmware beforehand, and that data is there. Also it is a firmware that is already in production and used on Android devices.
I am using a url to an online server, with the .bin and .dat files. Could this be an issue? I tried with the Zip and got errors saying that the zip could not be done (Error while creating ZIP stream: The operation couldn’t be completed. (Zip.ZipError error 0.)). All of the correct files and manifest are there. Thanks again.

We need to see some actual code or a symbolicated crash log to be able to help with those type of errors.
If you revert back to the latest version of the Pods release does the error go away ?
if not let us know, step through the code and see what exactly is giving you the EXC_BAD_ACCESS code so we can debug

The function gets called 3 times after the device is initially connected. The first time, the device enters bootloader mode. The second time, connects to the bootloader (dfuTarg). The third time, it starts the firmware update. self.selectedFirmware is the firmware taken from the website. Everything seems to execute fine except for the last step at initiator.withfirmware. Before, with the old revision, everything would work fine, but the update would be stuck on Connecting....
I hope this is everything you need, but if you need more or need me to explain more, please let me know. I am struggling with this and I am way past my deadline. Thanks so much for everything
`func DFUUpdateStart(){
print("Entered DFUUpdateStart function")
// (Step 1) If connected to a new 110 PLC and no DFUPeripheral has been selected and saved
if self.selectedPeripherals.count == 1 && self.DFUPeripheral == nil {
self.dfuInitiatorRan = false
self.dfuDelayRunAgain = true
let url = URL(string: "http://reverie.com/RnD/110_plc.bin")
let dtUrl = URL(string: "http://reverie.com/RnD/110_plc.dat")
self.selectedFirmwareDFU = DFUFirmware(urlToBinOrHexFile: url!, urlToDatFile: dtUrl, type: DFUFirmwareType.application)
//            let url = URL(string: "http://reverie.com/RnD/110_plc_sd110_dfu_1_03_10-5.zip")
//            self.selectedFirmwareDFU = DFUFirmware(urlToZipFile: url!)
if let characteristicArray = self.selectedCharacteristicsBTLE110[selectedPeripherals[0].identifier.uuidString] {
for characteristic in characteristicArray where characteristic.uuid.uuidString == REVERIE_110_DFU_UPDATE_CHARACTERISTIC_UUID {
print("DFU Characteristic is notifying: (characteristic.isNotifying)")
//                    let bytes:[UInt16] = [0x0401]
let bytes:[UInt8] = [0x01, 0x04]
let data = Data(bytes: UnsafePointer(bytes), count: bytes.count)
selectedPeripherals[0].writeValue(data, for: characteristic, type: .withResponse)
print("Writing to DFU to flag it manually for update.")
//TODO: Add check for implementation of delegate function
self.dfuStep = 1
print("Changed DFU Step to 1")
self.delegate?.dfuUpdate!(self.dfuStep, state: self.dfuState, part: self.dfuPart, totalParts: self.dfuTotalParts, progress: self.dfuProgress, currentSpeed: self.dfuCurrentSpeed, avgSpeed: self.dfuAvgSpeed, error: self.dfuError, errorMessage: self.dfuErrorMessage)
}
}
return
}
// (Step 2) once disconencted from PLC, scan for DFU Peripherals and if found, connect to it
if self.selectedPeripherals.count == 0 {
self.centralManager?.scanForPeripherals(withServices: [CBUUID.init(string: REVERIE_110_DFU_SERVICE_UUID)], options: nil)
print("Scanning for DFU peripheral")
if let peripheral = self.DFUPeripheral {
self.dfuDelayRunAgain = true
self.centralManager?.connect(peripheral, options: nil)
print("Saved DFU peripheral detected, connecting to it")
self.dfuStep = 2
print("Changed DFU Step to 2")
self.delegate?.dfuUpdate!(self.dfuStep, state: self.dfuState, part: self.dfuPart, totalParts: self.dfuTotalParts, progress: self.dfuProgress, currentSpeed: self.dfuCurrentSpeed, avgSpeed: self.dfuAvgSpeed, error: self.dfuError, errorMessage: self.dfuErrorMessage)
} else {
if self.dfuDelayRunAgain {
Timer.scheduledTimer(timeInterval: 1.5, target: self, selector: #selector(BTLEInterface.DFUUpdateStart), userInfo: nil, repeats: false)
self.dfuDelayRunAgain = false
print("DFU Update delay started 1.")
}
}
return
}
// (Step 3) if connected to a DFU Peripheral, update it.
if let peripheral = self.DFUPeripheral {
if let firmware = self.selectedFirmwareDFU {
if self.dfuDelayRunAgain {
Timer.scheduledTimer(timeInterval: 1.5, target: self, selector: #selector(BTLEInterface.DFUUpdateStart), userInfo: nil, repeats: false)
self.dfuDelayRunAgain = false
print("DFU Update delay started 2.")
} else {
print("Firmware Size: (firmware.size)   Parts: (firmware.parts)")
if self.centralManager?.retrieveConnectedPeripherals(withServices: [CBUUID(string: REVERIE_110_DFU_SERVICE_UUID)]).count != 1 {
Timer.scheduledTimer(timeInterval: 1.0, target: self, selector: #selector(BTLEInterface.DFUUpdateStart), userInfo: nil, repeats: false)
print("Refiring DFUUpdateStart because device is not connected yet")
} else if !self.dfuInitiatorRan {
print("Starting DFU Update")
//                        let initiator = DFUServiceInitiator(centralManager: self.centralManager!, target: peripheral).withFirmwareFile(firmware)
let initiator = DFUServiceInitiator(centralManager: self.centralManager!, target: peripheral)
initiator.with(firmware: firmware)
// Optional:
// initiator.forceDfu = true/false; // default false
// initiator.packetReceiptNotificationParameter = N; // default is 12
initiator.logger = self // - to get log info
initiator.delegate = self // - to be informed about current state and errors
initiator.progressDelegate = self // - to show progress bar
// initiator.peripheralSelector = ... // the default selector is used
                    initiator.start()
                    //        let controller = initiator.start()
                    self.dfuStep = 3
                    print("Changed DFU Step to 3")
                    self.delegate?.dfuUpdate!(self.dfuStep, state: self.dfuState, part: self.dfuPart, totalParts: self.dfuTotalParts, progress: self.dfuProgress, currentSpeed: self.dfuCurrentSpeed, avgSpeed: self.dfuAvgSpeed, error: self.dfuError, errorMessage: self.dfuErrorMessage)
                    self.dfuInitiatorRan = true
                }
            }
        } else {
            print("DFU Firmware is nil")
            self.dfuStep = 0
            print("Changed DFU Step to 0")
            self.dfuErrorMessage = "DFU Firmware is not loaded. Please try again."
            self.delegate?.dfuUpdate!(self.dfuStep, state: self.dfuState, part: self.dfuPart, totalParts: self.dfuTotalParts, progress: self.dfuProgress, currentSpeed: self.dfuCurrentSpeed, avgSpeed: self.dfuAvgSpeed, error: self.dfuError, errorMessage: self.dfuErrorMessage)
        }
        return
    }
}`

Perfect ! Itll be the first thing i look at tomorrow morning during work and will update you right away!

@sanderdb01 quick question, is that Secure DFU or Legacy DFU ?

I'm not sure. How can I tell if it is legacy or secure?

it's secure if it requires signing, you can also check the UUID of the DFU service, if it's FE59 then it's the secure one, if it's not then it's legacy

Yeah, looking at the DFU service UUID (00001530-1212-EFDE-1523-785FEABCD123), and I am 99% sure we are not doing signing, I think it is legacy then.

Perfect, I see you had the same Zip issue before here [11] (https://github.com/NordicSemiconductor/IOS-DFU-Library/issues/11), how did you fix that ?

I never did, I just used the .dat and .bin file instead. The issue was never resolved.

Hey guys, any update yet on the issue? I never heard anything back. Never made if further than telling you it was legacy.

I am attaching a screenshot of the error for your reference

So sorry about the late response, i saw another email that an issue was closed and thought it was this one so i never looked back!
your issue seems to be due to the Zip library, and since you never actually fixed the issue i linked earlier I'm pretty certain that your issue is with the Zip framework.
The fastest way I can help you is that if you create a small example project that uses the Zip cocoapod (make sure it's the exact same version as the one you are having in our iOSDFULibrary by checking the library's PodSpec file, under the dependencies, you'll find the version of the Zip Pod.
Then try to unzip one of the files that are failing, if it fails, please send me a link to the project so i can apply the fix as soon as possible!, if it doesn't reproduce, then let me know too!
again, please accept my apology for the late response

Hi, I was wondering if it was ok if i just sent you a link to our project on bitbucket, and you could just check our code there? Would that be easier?

Sure thing, if that's fine by you i'll gladly take a look on Bitbucket and send you a PR when i find the fix

ok great. I will push the latest version later on today and send you the link. Should I post it here or email it to you?

perfect, keep me updated, better keep it private via email, i'll shoot you an email now

Ok we have seem to found the issue, in your code you need to hold a reference to a DFUServiceController, you seem to have left that out on start.
so your line that says:
initiator.start()
needs to be:
self.dfuController = initiator.start()
and dfuController can be defined as a property in your view
var dfuController: DFUServiceController?
i gave it a try here and it worked, let me know how it works for you
My colleague @philips77 is also going to work on getting a fix so that you don't need to hold a reference that will come in version 3.0.2 soon.
@filippocamillo I see you had a similar issue, can you try this fix and see if it resolves it for you ?
thanks

Hi, I've tested the library and, as I expected, you don't have to hold any reference to make it work. I did test with the Example app where initiator is local and I removed assigning the controller and still DFU for Legacy and Secure works fine.
When DFU is started there are always 2 classes: executor and peripheral that hold a cyclic reference to one each other which is released on Complete, Abort or Error.

@sanderdb01 Question: why do you trigger the jump to bootloader on your own? The library does this automatically. You can start initialiser just after you have the DFUFirmware object.

@philips77 We connect to our peripheral normally, then we have to set a characteristic value to 1 to make it jump to the bootloader. Once this happens, the app automatically disconnected from the peripheral (because it is "no longer there"), and the peripheral turns into DFUTarg. I then Have to connect to DFUTarg to start the DFU process. I think this answered your question. If there is a better way of doing it, I am all ears lol.
So @philips77 and @mostafaberg , assigning the return value from the initiator does not make it work, right? I will still try it on my end, JIC. Any other ideas? Thanks a bunch.

@mostafaberg Also, did the ZIP issue ever get figured out? I still cannot get my URL for the zip to work. Thanks.

@mostafaberg @philips77 I just check the code with that change, and it still does not work. I should also mention, the error starts at the .with(Firmware) line, not the .start() line. So it happens before I get to start the process.

How about:
let url = URL(string: "http://reverie.com/RnD/110_plc.bin")
let dtUrl = URL(string: "http://reverie.com/RnD/110_plc.dat")
let firmware = DFUFirmware(urlToBinOrHexFile: url!, urlToDatFile: dtUrl, type: DFUFirmwareType.application)

print("Starting DFU Update")
let initiator = DFUServiceInitiator(centralManager: self.centralManager!, target: peripheral).with(firmware: firmware)
initiator.logger = self // - to get log info
initiator.delegate = self // - to be informed about current state and errors
initiator.progressDelegate = self // - to show progress bar
initiator.start()

Switching to DFU mode will be handled automatically by the library. No need to enable notifications and send 0x01-04 on your own. There is nothing custom there, as I see.

The peripheral does not have to be in DFU mode. It also can be connected if you held the connection before but it will disconnect at the end of the process so you need to reconnect to it when done/error (if needed).

@philips77  I talked to our firmware designer, and there is a bug in our module that makes it so that we HAVE to set the characteristic and reconnect. That was our issue, not yours.
Still, I am getting the same BAD_EXC on the .with function call. I am also getting an error in the terminal about the Zip module (the same as in the screenshot above). I am wondering if it is related.

I didnt check with zip i just used the code you had that uses the bin and dat files, i can debug the zip issue tomorrow and update you !

@mostafaberg @philips77  I had a thought. Is it possible that perhaps I have a build error? I know it was mentioned that I have (somehow) several copies of the framework. Maybe that is the issue? Is there a way that I can remove everything related to the iOSDFULibrary, then reinstall it? How would I go about doing that and not mess anything up? Thanks.

Its quite simple, run pod deintegrate this will remove all cocoapods related files, then delete any extra files that you see related to the library from the filesystem, then delete any remaining files or framework fiels from xcode, after all is cleared update to the latest version of cocoapods sudo gem install cocoapods and finally reinstall the pods pod install and you should be set

I have exactly the same issue. My environment is Xcode 8.2.1 and my app is in Objective-c. I rewrote a little bit the way I start the upload but it stuck on connecting.
 DFUFirmware *newFirmware = [[DFUFirmware alloc] initWithUrlToZipFile:url];

 _initiator = [[[DFUServiceInitiator alloc] initWithCentralManager:_centralManager
                                                               target:_peripheral] withFirmware:newFirmware];
 _initiator.logger = self; // - to get log info
 _initiator.delegate = self; // - to be informed about current state and errors
 _initiator.progressDelegate = self;

 _controller = [_initiator start];

I just have a doubt, do I need to use the experimental_ble_app_buttonless_dfu (or similar architecture in my own app) to make the library working?
Thx

@ArduinoManager Your code seems correct, but seems like you're connecting to a peripheral, jumping to bootloader your self, then using that same _peripheral in the DFU process, which will not work, after switching a peripheral to DFU/Bootloader mode, it starts broadcasting as a new peripheral with a new identifier, you'll need to rescan for it and add reassign it to _peripheral before starting DFU.
However, the library does the DFU/Bootloader jump for you, and yes, that is what experimental_ble_app_buttonless_dfu does
I would recommend simply doing the following and see if this helps :

Scan for target
Assign it to your _peripheral variable
Create a DFUServiceInitiator with your _peripheral, and set the experimental_ble_app_buttonless_dfu flag.
Start the initiator

If you have a reason to do the DFU jump yourself, you just scan and find the new peripheral after it jumps to DFU, usually named DFUTarg and broadcasts the DFU / Secure DFU service UUID.

@ArduinoManager what Mustafa wrote applies to SDK 12 and Secure DFU only. If you are using Legacy DFU (SDK before 12) the buttonless jump feature is demonstrated for example in HRM_DFU sample in the SDK. There the device after switching to DFU advertises with the same address, so you don't have to scan on your own. Actually, the lib will also switch to BL mode automatically when you use the standard impl.
If you use SDK 12 and Secure DFU and the experimental buttonless feature for Secure DFU you may set the flag to true in the DFUServiceInitiator and lib will also jump to BL for you. As Mustafa wrote, the only case when you have to trigger switch to BL and scan on your own is when you have a proprietary solution for this, for example you use your own service UUIDs, etc.

@mostafaberg Thank you for your valuable support. Because when the device is in DFU mode, the peripheral changes, scanning again and getting a new _peripheral object was the solution. Right now, I manually switch the device in DFU mode because I don't know yet if my final device will implement the DFU service. I'll try experimental_ble_app_buttonless_dfu out anyway. It would be great if the library returned an error because it is not able to get connected to the device instead of stuck without any error. That said, integrating the library was quite smooth.
philips77 Thank you for the clarification. I am using SDK12.

@philips77 @mostafaberg
I am back from the holiday and I tried uninstalling the library and then reinstalling it to see if it worked, but it still does not. I am still getting a fail at initiator.with(firmware: firmware) with an error of Thread 1: EXC_BAD ACCESS (code=1, address=0x10)
Has there been any update on the Zip not working?
Thanks a bunch

HI, I also noticed that the Zip.framework text in the navigation tree is in red. I assume this is a problem having to do with the file locations. I however didnt change anything. COuld this be the problem? And if so, how do I fix it? Thanks.

@sanderdb01 It's a bit unclear on why you're having this issue, but as I said earlier, Zip seems to be working fine for us, I can take a look if you email me one of the zip packages that's crashing for you and see if it works on our side maybe ? might rule out one issue at least that the Zip is in fact ok

@mostafaberg I believe I sent a zipped copy of our project before for you guys to test, but I am unclear on how the results of those tests went (if you got it to work or not). I can send our version of just the Zip package if you want. How would you want me to go about that (should I zip up the zip folders, and if so where in the project tree should I get it)?
I was wondering if perhaps we could skype or get on a phone call to resolve this just to make sure we are on the same page. This was a hot topic some time ago, and it is a big emergency now. Thanks a bunch for everything.

@sanderdb01 I mean the Zip package and not the whole project, regarding the last time when i tested the project I have responded here as you see, it did work and i was able to run the DFU process and sent you the changes required to get it to work, However it didn't work out on your side. I didn't manage to test it with the distribution package Zip file. so If you can send me the Actual distribution package, i can test locally and see if it's actually the problem or not.
I would still recommend updating the cocoapods installation and update to the latest version of the library in any case
We can get in touch by email to discuss your issue further ! you still have my email ?

@mostafaberg I tried assigning the return value and it still didn't work (referring to your response from before. I thought I had mentioned it, but it turns out I did not, I am sorry about that. I also didnt realize that you were able to get my program's DFU functionality working, I must have missed that as well or misunderstood. Very sorry about all the confusion.
I am uncertain what you mean by the zip package. I am assuming you mean the zipped file of the new firmware. If that is the case, it is located at the url: http://reverie.com/RnD/110_plc_sd110_dfu_1_03_10-5.zip
The zip file seems to work on the nordic app as well as Android, so I am thinking that the zipped firmware is not the issue.
If you mean the Zip Library files that are part of the project, let me know how you would want me to send that to you.
I also want to reiterate that I am getting the failure at: initiator.with(firmware: firmware) with an error of Thread 1: EXC_BAD ACCESS (code=1, address=0x10). I am currently using the .dat/.bin DFUFirmware initializer
(example: let url = URL(string: "http://reverie.com/RnD/110_plc.bin")
let dtUrl = URL(string: "http://reverie.com/RnD/110_plc.dat")
self.selectedFirmwareDFU = DFUFirmware(urlToBinOrHexFile: url!, urlToDatFile: dtUrl, type: DFUFirmwareType.application))
I also tried changing the DFUFirmwareType field to all available options, and they all had the same result.
If there is anything else that I can provide that will help, please let me know. Thanks a bunch.
I want to also add what my Podfile.lock file looks like. I just want to make sure that I have the correct version of the Zip library:
PODS:

iOSDFULibrary (3.0.0):

Zip (~> 0.6)

Zip (0.6.0)

DEPENDENCIES:

iOSDFULibrary

SPEC CHECKSUMS:
iOSDFULibrary: 201dec70ae2d7e1cfbdc3cd8005b163f009c9dea
Zip: 805fc2fa9d2f05bbb7762d982d7a42ccdcc51f42
PODFILE CHECKSUM: 7b25d8e1e32a48733e6372f90cd842d1ec96c1eb
COCOAPODS: 1.1.1

@mostafaberg
I wanted to add to my above post after I ran some tests. Another issue I am finding has to do with the execution of the with(Firmware) function. That is where I was getting the EXC_BAD error. In an attempt to hunt the issue down, I was trying to figure out why I was getting the error, and it was point to a null pointer. So, I then tried to add a new function to the DFUServiceInitalizer class called test(), and all it did was print "test". When I then tried to execute that function on the newly initialized DFUServiceInitializer object, I got the same EXC_BAD error. I then added a testString variable to the DFUServiceInitializer class (var testString:String = ""), and then after running the DFUServiceInitalizer init function (per usual), I then tried to set the testString variable just to see if I could access the initiator variable at all, and still got the EXC_BAD error.
So, it seems that the program fails with the EXC_BAD error whenever I try and access the initiator variable (DFUServiceIntializer object).
I was not getting this error before the conversion to Swift3, I wanted to make sure I was mentioning that. Thanks again.

@sanderdb01 Thanks for all the input !, I have everything I need now so i'll be debugging a bit today and will let you know as soon as possible how it goes, if i get things working i'll just send you a working copy of the project later in the day!

@sanderdb01 Ok I got things working, I pushed a new branch on your repository bugfix/dfu with a working version, make sure to check for the //HACK comments where i had to modify minor parts of your code to be able to connect to any peripheral in DFU directly, I'll shoot you an email now too with some info!

@sanderdb01 This works now after the fix, right ? I'll close by the end of the day if i don't hear back, or feel free to close it if you read this earlier, or update me with the status so we can fix anything that's remaining!

Hello. Sorry to open again this issue but I am having the same and I am not able to fix it. Today I updated the iOS DFU Library and I am getting the EXC_BAD_ACCESS (code=1, address=0x10). Before the update everything was working properly.
I also tried removing and reinstalling all the pods from scratch.
Here is my code:
func setFilePath(){
        let bundleURL = Bundle.main.bundleURL
        let dataFolderURL = bundleURL.appendingPathComponent("firmware")
        let fileURL = dataFolderURL.appendingPathComponent("edd_new.hex")

        let fileManager = FileManager.default
        if fileManager.fileExists(atPath: fileURL.path) {
            print("FILE AVAILABLE")
        } else {
            print("FILE NOT AVAILABLE")
        }

        let selectedFirmware = DFUFirmware(urlToBinOrHexFile: fileURL, urlToDatFile: nil, type: DFUFirmwareType.application)

        startUpdate(selectedFirmware!)
    }

    func startUpdate(_ fw: DFUFirmware){

        if(self.dfuPeripheral == nil){
            print("DFU PERIPHERAL NIL")
        }

        let initiator = DFUServiceInitiator(centralManager: centralManager!, target: self.dfuPeripheral!)

        initiator.logger = self; // - to get log info
        initiator.delegate = self; // - to be informed about current state and errors
        initiator.progressDelegate = self; // - to show progress bar

        self.dfuController = initiator.with(firmware: fw).start()
    }

File is available, and peripheral is not nil. It also crashes on the .with(firmware: fw) part. Tried both to put in on the initiator declaration and before the start as the code shows. Same problem.
Could you please share with me the solution? I am a bit in a rush. Thank you!

@mostafaberg
I am running into the same problem. For some reason, as soon as it executes the DFUFirmware(URL) constructor, it gives me the error: "Error while creating ZIP stream: The operation couldn't be completed."
Here is my code:
let documentsUrl:URL =  (FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first as URL?)!
        let destinationFileUrl = documentsUrl.appendingPathComponent(Constant.firmwareLocalPath)

        let selectedFirmware = DFUFirmware(urlToZipFile: destinationFileUrl)

        let initiator = DFUServiceInitiator(centralManager: self.central, target: self.peripheral).with(firmware: selectedFirmware!)
        initiator.logger = self; // - to get log info
        initiator.delegate = self; // - to be informed about current state and errors
        initiator.progressDelegate = self; // - to show progress bar
        initiator.enableUnsafeExperimentalButtonlessServiceInSecureDfu = true

        self.dfuServiceController = initiator.start()

and my firmware path looks like this:
file:///var/mobile/Containers/Data/Application/64D07EC7-A64F-4D7B-9F89-7A5A700CAD5A/Documents/firmware.zip
A quick reply will be appreciated. I am not sure why it is behaving like this? To be clear, nothing is nil. Central, Peripheral and path, everything is valid.

I resolved my "Error while creating ZIP stream: The operation couldn't be completed."
It seems that the iOS ZipArchive.unzip(urlToZipFile) create by default two folders : including the below in DFUStreamZip.swift solved my problem.
init(urlToZipFile: URL, type: DFUFirmwareType) throws {
        // Try to unzip the file. This may throw an exception
        var contentUrls = try ZipArchive.unzip(urlToZipFile)

//----- INCLUDE ----
        let file_name = urlToZipFile.deletingPathExtension().lastPathComponent
        for url in contentUrls {
            if url.lastPathComponent == file_name {
                contentUrls = try FileManager.default.contentsOfDirectory(at: url, includingPropertiesForKeys: nil,options: [])
                break
            }
        }
        //TEST: List zip archive content
        //for url in contentUrls { print(url.absoluteString) }
//---- END INCLUDE ----

       // Look for MANIFEST_FILE
.
.
.


Remove string-based reference of tokens in CodeGeneration

CodeGeneration currently refers to tokens e.g. as IdentifierToken. That string-based reference isn’t very idiomatic Swift. What we should have instead is a Token enum of all the possible tokens in TokenSpec.swift, a computed property on that enum that returns the corresponding TokenSpec of that token. We could then use members of the Token enum to refer to the token kinds, instead of the string-based reference.

Tracked in Apple’s issue tracker as rdar://113237461

This issue sounds like really amazing improvement. I really like the idea and I can implement it with great pleasure. Just to clarify: we want to replace refers to every token with this new enum, not only SYNTAX_TOKENS?

Yes, exactly. For example
https://github.com/apple/swift-syntax/blob/250b431726c7b05309770caaa9989a4166ea6ba7/CodeGeneration/Sources/SyntaxSupport/AvailabilityNodes.swift#L33
should be something like
choices: [.token(tokenKind: .binaryOperator), .token(tokenKind: .identifier)],

And also just to clarify: This is a CodeGeneration-only change. The generated files should not change.


When I try to Mixtral 7B Full finetuning, I got an error message about  Flash Attention 2.

Please check that this issue hasn't been reported before.

[X] I searched previous Bug Reports didn't find any similar reports.

Expected Behavior
I got a this error when I try to full fine-tuning mixtral 7bx8
Current behaviour
File "/home/jovyan/fileviewer/LLM/axolotl/src/axolotl/utils/models.py", line 453, in load_model
model = AutoModelForCausalLM.from_pretrained(
File "/home/jovyan/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
return model_class.from_pretrained(
File "/home/jovyan/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3444, in from_pretrained
config = cls._autoset_attn_implementation(
File "/home/jovyan/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1302, in _autoset_attn_implementation
cls._check_and_enable_flash_attn_2(
File "/home/jovyan/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1401, in _check_and_enable_flash_attn_2
raise ImportError(f"{preface} Flash Attention 2 is not available. {install_message}")
ImportError: FlashAttention2 has been toggled on, but it cannot be used due to the following error: Flash Attention 2 is not available. Please refer to the documentation of https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 to install Flash Attention 2.
Steps to reproduce
I install your package for full-finetuning mixtral 7B x 8.
However, I encountered above issue.
How can I solve it?
Config yaml
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true
load_in_8bit: false
load_in_4bit: false
strict: false
datasets:

path: DopeorNope/Robustness_Ko_data-v1
type: alpaca
dataset_prepared_path: last_run_prepared
val_set_size: 0.0
output_dir: ./mixtral-ko-v1

You can optionally freeze the entire model and unfreeze a subset of parameters
unfrozen_parameters:
- lm_head.*
- model.embed_tokens.*
- model.layers.2[0-9]+.block_sparse_moe.gate.*
- model.layers.2[0-9]+.block_sparse_moe.experts.*
- model.layers.3[0-9]+.block_sparse_moe.gate.*
- model.layers.3[0-9]+.block_sparse_moe.experts.*
sequence_len: 32768
sample_packing: true
pad_to_sequence_len: true
adapter:
lora_model_dir:
lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:
#lora_target_modules:
#  - gate
#  - q_proj
#  - k_proj
#  - v_proj
#  - o_proj
#  - w1
#  - w2
- w3
adapter:
lora_model_dir:
lora_r:
lora_alpha:
lora_dropout:
lora_target_linear:
lora_fan_in_fan_out:
wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:
wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:
gradient_accumulation_steps: 64
micro_batch_size: 1
num_epochs: 5
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: false
group_by_length: false
bf16: false
fp16: True
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: True
loss_watchdog_threshold: 5.0
loss_watchdog_patience: 3
warmup_steps: 10
evals_per_epoch: 4
eval_table_size:
eval_table_max_new_tokens: 128
saves_per_epoch: 1
debug:
deepspeed: deepspeed/zero2.json
weight_decay: 0.1
fsdp:
fsdp_config:
special_tokens:
Possible solution
No response
Which Operating Systems are you using?

[X] Linux
[ ] macOS
[ ] Windows

Python Version
3.10
axolotl branch-commit
rescent
Acknowledgements

[X] My issue title is concise, descriptive, and in title casing.
[X] I have searched the existing issues to make sure this bug has not been reported yet.
[X] I am using the latest version of axolotl.
[X] I have provided enough information for the maintainers to reproduce and diagnose the issue.

Can you provide the output of running pip list ?

Please also check that nvidia-smi shows your GPU and you properly set CUDA_VISIBLE_DEVICES.

Should be solved by re-creating new env and re-running following readme. Closing.


Update Storybook

We are currently using Storybook 6.4.22 - the latest is 7.5.1. We should see what all is involved in upgrading and see if this is possible.

Merged into templates branch.


version <IP_ADDRESS> Nuget vs github dll wrong

They Are not the same....
github <IP_ADDRESS> - works
nuget <IP_ADDRESS> - not working
dll both say <IP_ADDRESS>
what is not working:
import xls file with IsFirstRowHeaderRow = false
This row will not appear in nuget dll but will in github.
github version is correct as far as i am concerned.

Don't worry, just use latest develop build.

All unintended binaries are removed from the develop branch. Not sure what happened with <IP_ADDRESS> but the one on Nuget.org is the official one that everyone gets


Improve test suite towards v2.0.0 release

Test suite needs full coverage of all lines and errors for releasing v2.0.0 of the code and bringing it out of the development branch.
Note: The GUI part continues to not be tested at all.
All jitted functions need to be fully tested, however they are currently still excluded from coverage since jitting and coverage don't work together. Coverage is asserted manually.

Some lines in utilities/peirce.py are uncovered. Can think of doing better once the Peirce criterion is actually out of the testing phase... In fact, I should add a warning to the peirce filter that it is largely untested...