use local gpu

how to use local GPU in TabNine Local Mode?

Sorry, we don't currently support using the local GPU with TabNine Local. It's added to the backlog.

Looking forward to your good news

same <3

Really need this feature. CPU load causes fans to make very loud noises.

Any news on this? Would be great to not have CPU fan running any time I try to type :)

hope to support

would be great to have this feature, any news?

would be great to have this feature, any news?

no, maybe this is a complex project.

@lujfsd Tabnine actually offers GPU computation on their cloud: https://www.tabnine.com/subscribe/#cloud so it seems to me, there is existing and working code, to run Tabnine on a GPU.
I have a more than good enough GPU in my local machine, thus I'd be very interested in a feature, to use my local GPU instead of a remote server.

why not thinking from the beginning @tabnine to make the AI computation on the user GPU or put the user into the position to choose between CPU or GPU in order to achieve the best user experience from performance point of view ??? i have a GTX 1050 on my laptop on linux and with tabnine its NOT used at all. its waste of power... my CPU is spiking all the time when i type 1 character because your great local machine learning algorithm its not executed on cuda cores (in case of nvidia), instead its executed on a poor CPU. great. thanks. i mean you train your neural nets on GPUs ... cmon guys


feat: create env page

Small tweak to align the New buttons with search bar & tweak the + symbol to match others

Small design tweaks
• Made save button fit width of words
• Bolded important text in radio button paragraphs
• Made the radio button label text bigger


Fine-tune on Middlebury and ETH3D

Hi,
Thanks for sharing the code.
I found this method also participated in the Robust Vision Challenge and have submissions to both datasets.
May I ask what is the implementation details for that version?
For example, did you also pre-Train the model on Sceneflow and fine-tune on each dataset separately with 300 epochs, or maybe fine-tune on the combined 3 datasets with a certain schedule?
Thanks

@fuy34
For the Challenge, we fine-tune the model on the combined 3 datasets.


Regression: some types are misplaced or not displayed

Search terms
types misplaced not displayed
Actual Behavior
I have this code:
export const BODY_TYPES = {
  DYNAMIC: 1,
  STATIC: 2,
  KINEMATIC: 4,
} as const

export type BodyType = typeof BODY_TYPES[keyof typeof BODY_TYPES]

export class Body  {
   /**
   * One of: `Body.DYNAMIC`, `Body.STATIC` and `Body.KINEMATIC`.
   */
  type: BodyType

  // ...
  constructor(
    options: {
      /**
       * One of: `Body.DYNAMIC`, `Body.STATIC` and `Body.KINEMATIC`.
       */
      type?: BodyType

      // ...
    } = {}
  ) {

}

And the options.type is not showing the BodyType, but instead its literal value (happens from v0.20.6, possibly related to #1528).

While the class property, is showing totally a different type!! (happens from v0.20.13)

For context, the type RayMode is in totally another part of the application and corresponds to this:
export const RAY_MODES = {
  CLOSEST: 1,
  ANY: 2,
  ALL: 4,
} as const

export type RayMode = typeof RAY_MODES[keyof typeof RAY_MODES]

Expected Behavior
This should be the correct behaviour as it is with v0.20.5:

Steps to reproduce the bug
I can link the repo where this happens if needed
Environment

Typedoc version: v0.20.6 and up

What TypeScript version are you using?

4.0.5, but I'll try upgrading it and see if the issue persists!

Updating typescript completely solves this issue, thanks!


[Bug] Macro added twice during partial parsing

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
While in main/master branch, received compilation error:
dbt found two macros named "<user_macro>" in the project "<user_project>"

To fix this error, rename or remove one of the following macros:
- /macros/foo/bar.sql
- /macros/foo/bar.sql

The error points to the same file twice, seems like the macro in question is duplicated within that file. Could not locate the duplicated entry in the file. The *.yml file was clear of duplicates as well.
Expected Behavior
No compiler errors, possibly 3 warnings for this user.
Steps To Reproduce
Possible user steps:

Create new file with 2 macros while in development environment and under new branch.
Add both macros to macros yml file.
Save commits and switch to main branch.
Compilation error is raised.

Forced error steps:

Create new file with 2 macros while in development environment and under new branch.
Add only one of the macros to macros yml file.
Compilation error is raised.

Relevant log output
Error was seen during partial parsing compilation

Environment
- dbt: 0.21.0

What database are you using dbt with?
redshift
Additional Context
No response

This was automatically closed as "stale" on Sep 17, 2022. Based on the report in https://github.com/dbt-labs/dbt-core/issues/8775, it was not resolved by https://github.com/dbt-labs/dbt-core/issues/4771.

Re-opening.
Reprex as reported in #8775:
1. Create `macros/macros.sql` file with 2 macros inside
2. Create `macros/macros.yml` with documentation for these 2 macros
3. Run `dbt compile`
4. Change something in `macros/macros.yml`
5. Run `dbt compile` again

Implementation hints
https://github.com/dbt-labs/dbt-core/pull/4773 might have some ideas for implementation.

This issue still exists. I can reproduce it the same way OP did.

Thank you for confirming @pcgarciar
Does it resolve the error for you if you use the --no-partial-parse flag?
dbt compile --no-partial-parse

This the workaround that that we've been recommending in the meantime for this type of error message.

Hello @dbeatty10 ! I am also seeing this same issue after upgrading my dbt  version to 1.6.17 .
There is a problem in your dbt project. Compilation failed: Compilation Error   dbt found two macros named "materialization_materialized_view_default" in the   project "dbt".    To fix this error, rename or remove one of the following macros:       - macros/materializations/models/materialized_view/materialized_view.sql       - macros/materializations/models/materialized_view.sql
 However I cannot locate these filepaths in my project at all. The recommended workaround

  dbt compile --no-partial-parse did not fix it.

@jasmin-hemdani What is the output of dbt --version? That way, I can see all the dbt adapter plugins you have installed (and their versions).
Do you have any dbt packages installed via dbt deps? If so, could you share the contents of your relevant packages.yml / dependencies.yml file?

Thank you for confirming @pcgarciar
Does it resolve the error for you if you use the --no-partial-parse flag?
dbt compile --no-partial-parse

This the workaround that that we've been recommending in the meantime for this type of error message.

Unfortunately that does not seem to fix it, but I found that deleting manifest.json does. However I'm scared that doing this might break other stuff (although that doesn't appear to be the case).

Hello @dbeatty10 ! I am also seeing this same issue after upgrading my dbt version to 1.6.17 . There is a problem in your dbt project. Compilation failed: Compilation Error dbt found two macros named "materialization_materialized_view_default" in the project "dbt". To fix this error, rename or remove one of the following macros: - macros/materializations/models/materialized_view/materialized_view.sql - macros/materializations/models/materialized_view.sql However I cannot locate these filepaths in my project at all. The recommended workaround  dbt compile --no-partial-parse did not fix it.

I have the same issue... any idea?

@sgalban-WTW Does it resolve the error for you if you use the --no-partial-parse flag?
dbt compile --no-partial-parse

If not, does it resolve if you delete your manifest.json file?
There's a couple ways to do that:

dbt clean
rm target/manifest.json (or whatever the path to your manifest file is)

HI! It still not working with this flag.
I have just deleted the dbt-adapters from my venv and it might be work

@sgalban-WTW I wouldn't expect that deleting dbt-adapters from your virtual environment would affect this one way or the other.
Did you try deleting your target directory and trying again? It is safe to delete your target directory because dbt will just recalculate what it needs from scratch.

Hi Team, adding on here as the issue is still around. dbt Cloud customer reported seeing the error happen a few times, albeit randomly. The error seems to only pop up when editing the macros.yml file.
The current workaround is to:

Run dbt parse --no-partial-parse
Or delete the target/ folder and then refresh Cloud IDE

I was able to reproduce it under certain conditions (similar to the comment here):

Have multiple macros in macros.sql
Edit/add macros with similar names in macros.yml

It is rather elusive though, sometimes reproducible a few times in a row with the same steps, other times, nothing happens.
For me, the error always clears by deleting the target/ folder.
The issue has been around since 2021 and it is not always reproducible - is there anything that we can help with / provide in order to help the troubleshoot?

Hey @wusanny !
My understanding is that deleting the target folder is a reliable way to clear the error.
I don't know of anything else that you can help with or provide.
Here's the way I've reproduced this error previously: https://github.com/dbt-labs/dbt-core/issues/4233#issuecomment-1749535081
And here's some potential implementation hints: https://github.com/dbt-labs/dbt-core/issues/4233#issuecomment-1749540702


Support validation of non-standard inputs and add validation for TicketCreate

Sorry about the noise of prettier formatting changes.
resolves #457

I'm not seeing it:


rethink index in run_benchmark()

perhaps it can "override" the index in approximators
sv_approximators = [
    shapiq.PermutationSamplingSII(n=n_players, index="k-SII", random_state=0),
    shapiq.SHAPIQ(n=n_players, random_state=0),
    shapiq.SVARMIQ(n=n_players, random_state=0),
    shapiq.KernelSHAPIQ(n=n_players, random_state=0)
]
# run the benchmark with the chosen parameters
results = shapiq.games.benchmark.run_benchmark(
    index="k-SII",
    order=2,
    games=games,
    approximators=sv_approximators,
    save_path="benchmark_results.json",
    budget_steps=[500, 1000, 2000, 4000],
    n_jobs=8
)

This is not really possible. The benchmark has multiple ways of instantiating and using approximators. They can be specified via a str, class, or already instantiated object. For a more flexible usage (e.g. using approximators for different indices that they are intended for) we leave the logic as is: An index must always be specified.


relu6 when convert tensorflow to caffe

Platform (like ubuntu 16.04/win10):
ubuntu1604
Python version:
3.5.2
Source framework with version (like Tensorflow 1.4.1 with GPU):
tensorflow
Destination framework with version (like CNTK 2.3 with GPU):
caffe
Pre-trained model path (webpath or webdisk path):
Running scripts:
Hello, everyone.
I have a question about relu6 of tensorflow.
The code is here.
def emit_Relu6(self, IR_node):
    self.emit_Relu(IR_node)

So, is relu6 the same as relu?

@hnuhchen , thank you very much for the feedback. Which version of MMdnn are you using? The emit_Relu6() in the latest 0.2.5 release is as follows:
# mmdnn/conversion/tensorflow/tensorflow_emitter.py, line 443
def emit_Relu6(self, IR_node):
        return self._emit_unary_operation(IR_node, 'nn.relu6')

It seems that we could use the Clip layer to simulate ReLu6. Strangely, such layer is not listed in the documentation of Caffe layers.

@hnuhchen , thank you very much for the feedback. ReLu6 computes rectified linear 6: min(max(features, 0), 6), which is not the same as ReLu. However, Caffe does not implement ReLu6 yet, so we use ReLu instead.

Thank you for your answer.
The output of relu is not the same as the output of relu6, so the modification may cause problems of consistence

@hnuhchen, it seems that we could use the Clip layer to simulate ReLu6. Please help test the pull request. Strangely, the Clip layer is not listed in the documentation of Caffe layers.
Thank you.
Unfortunately, my plateform does not support clip layer now, i cannot test your pull request.


Update words.py

#Hacktoberfest

What you have done, explain

What you have done, explain

I updated some words in code of your repository 🥺🥺

No, Makes no sense.


Plugin does not recognize parsedVersion variables

Is this a request for help?: No

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Environment (plugin version, maven version, OS, ...):
Plugin version: 6.3.0
Maven version: 3.6.3
OS: Ubuntu 20.04.4 LTS
What happened:
When I run mvn clean helm:package -DskipTests  I get the following error:
    [INFO] Setting chart version to ${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.incrementalVersion}
    [INFO] Setting App version to 1.3.0-SNAPSHOT
    [ERROR] Error: Invalid Semantic Version

What you expected to happen:
I expected that the variable names would be substituted for their values.
How to reproduce it (as minimally and precisely as possible):
Add and configure the maven help plugin like this:
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>build-helper-maven-plugin</artifactId>
				<version>1.8</version>
				<executions>
				  <execution>
					<id>parse-version</id>
					<goals>
					  <goal>parse-version</goal>
					</goals>
				  </execution>
				</executions>
			</plugin>

Configure the helm-maven-plugin like this:
			<plugin>
				<groupId>io.kokuwa.maven</groupId>
				<artifactId>helm-maven-plugin</artifactId>
				<version>6.3.0</version>
				<configuration>
					<useLocalHelmBinary>true</useLocalHelmBinary>
					<chartDirectory>${project.basedir}/src/main/helm/device-api</chartDirectory>
				  <chartVersion>${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.incrementalVersion}</chartVersion>
				  <appVersion>${project.version}</appVersion>
				</configuration>
			  </plugin>

Then run: mvn clean helm:package -DskipTests
Anything else we need to know:
I use the parsedVersions so that when working with SNAPSHOTS my chart version will be valid semver.

Try with the latest version of build-helper-maven-plugin - 3.3.0
Investigate output of mvn -X ...  and look how helm plugin is setup by Maven.

@swright-eti is this still a problem?

This has been resolved. Thank you.


Unnecessary try/catch?

Provide your feedback here.
https://github.com/adobe/react-spectrum/blob/fa11f1726881eaa5e73bf43178b57161ba2c61e9/packages/%40react-aria/i18n/src/useDefaultLocale.ts#L30
I'm looking at this code and part of it doesn't make sense. Why do you have the try catch there if the last check is setting locale to en-US. I could be reading this code wrong, but from what I gather from it, if you are doing SSR (which is why I'm assuming it could be undefined for navigator and language). then it would miss the first check and then default to en-US.
But even at that point couldn't you do
// But again if i'm reading this right, the locale should never be undefined or null
if(locale === undefined) {
  return {
    locale,
    direction: isRTL(locale) ? 'rtl' : 'ltr'
  };
}
if(Intl.DateTimeFormat.supportedLocalesOf([locale]).length === 0) {
  locale = 'en-US'
}
return {
    locale,
    direction: isRTL(locale) ? 'rtl' : 'ltr'
  };

🔦 Context
This hasn't really affected me per se. But I had a developer who copied this code and it didn't make sense to me. Especially as far as I've always known try/catch is somewhat of an anti-pattern for performance with JS when we could easily just check for undefined.
💻 Code Sample
No response
Version
No response
What browsers are you seeing the problem on?
No response
If other, please specify
No response
What operating system are you using?
No response

Thanks for the issue. Try/catch should generally be avoided. However, in this case, an error can be thrown which we aren't expecting.
I think the description of the PR adding it may address your question https://github.com/adobe/react-spectrum/pull/3030 Though we may have enough browser support for the alternative mentioned lower in the PR. We could look into removing it that way instead.

@snowystinger I'm getting "Incorrect locale information provided" error on isRTL() helper

@dbritto-dev I'm not sure what that has to do with this Issue?
Could you open a new issue with a codesandbox or stackblitz reproducing your issue?

@snowystinger sure


Inconsistencies between the JSON schemas and the specification

I found some inconsistencies between the JSON schemas and the specification.
Can you please clarify which one is correct so I can submit a fix?
Spec version: 0.8
workflow.json

Property
Schema
Specification

id
Required
Required if key not defined

name
Required
Not required

version
Required
Not required

states/switchstate.json

Property
Schema
Specification

default
Required
defaultCondition is required. Seems to be a typo

switchconditions/eventcondition.json

Property
Schema
Specification

transition
Required
transition (x)or end is required

branches/branch.json

Property
Schema
Specification

actions
Not required
Required

states/foreach.json

Property
Schema
Specification

actions
Not required
Required

events/onevents.json

Property
Schema
Specification

actions
Not required
Required

error/error.json

Property
Schema
Specification

error
Required
error is not a property

transition
Required
transition or end is required

retry/retrydef.json

Property
Schema
Specification

maxAttempts
Required
Not required

end/continueas.json

Property
Schema
Specification

kind
Required
Not a property

the schemas in the spec are the source of truth
what should be changed is that schema validation should use the schemas deployed on our website rather then internal:
https://github.com/serverlessworkflow/sdk-java/blob/main/api/src/main/java/io/serverlessworkflow/api/validation/WorkflowSchemaLoader.java#L28
would suggest only making this change
the schemas in sdk are only there for code generation and the required fields play no difference at all (you could remove all required elements completely and the code gen should work same.
so i would not worry about keeping the schemas required params in tact, but update the schema validation code to use spec schemas instead

@tsurdilo Im glad you mention this, because there is an issue I wanted to discuss with you some time ago.
As you are aware, there are some POJOs that are generated from the schema and others that are hardcoded. That is the reason why the workflow.json is copied into the SDK.
In particular, the issue seems to be related with the handling of oneOf in the schema
For example, for Events, in the spec we have
"events": {
      "$ref": "events.json#/events"
    },

and
"events": {
    "oneOf": [
      {
        "type": "string",
        "format": "uri",
        "description": "URI to a resource containing event definitions (json or yaml)"
      },
      {
        "type": "array",
        "description": "Workflow CloudEvent definitions. Defines CloudEvents that can be consumed or produced",
        "items": {
          "type": "object",
          "$ref": "#/definitions/eventdef"
        },
        "additionalItems": false,
        "minItems": 1
      }
    ]
  }

And in the SDK we have
events": {
      "type": "object",
      "existingJavaType": "io.serverlessworkflow.api.workflow.Events",
      "description": "Workflow event definitions"
    }

and
public class Events {
  private String refValue;
  private List<EventDefinition> eventDefs;

  public Events() {}

  public Events(List<EventDefinition> eventDefs) {
    this.eventDefs = eventDefs;
  }

  public Events(String refValue) {
    this.refValue = refValue;
  }

  public String getRefValue() {
    return refValue;
  }

  public void setRefValue(String refValue) {
    this.refValue = refValue;
  }

  public List<EventDefinition> getEventDefs() {
    return eventDefs;
  }

  public void setEventDefs(List<EventDefinition> eventDefs) {
    this.eventDefs = eventDefs;
  }
}

The problem is that the library we are using https://github.com/joelittlejohn/jsonschema2pojo does not handle oneOfas a union like (like our generated classes) but as an Object
I would like to remote that limitation by improving the generator and use the spec files directly to generate the code we want to have.
So, we will not have any hardcode POJO like the ones here https://github.com/serverlessworkflow/sdk-java/tree/main/api/src/main/java/io/serverlessworkflow/api/workflow. The idea is to change the generator to generate them from the spec (this implies either change the generator or use a custom one)
Since this change in the generator is not trivial , I want to achieve consensus before starting.
So, wdyt?

@fjtirado if you think we can improve the generation to use the spec schemas that sounds nice, maybe you can start this work in a separate branch and let's see where it takes us as I assume this might be a larger effort.
Every SDK has its little things that are different which is imo ok.
Limitations that are present due to gen tools I think can be masked very often by improving validation, so yes definitely what you propose now is imo a great idea but I don't think its something super urgent to do, so take your time no pressure at all.
We can/should look at improving validation (not so much its current structure, but more so adding more validation points) to help users as well as again mask the current sdk limitations.

@tsurdilo Great, Ill try and see where it goes.
The target of the effort is precisely to improve validation. If all POJOs are generated from the schema, then we can add javax validation annotations based on the schema in an uniform way, and, activating those annotations, we will have 95% of all the validation we need (implementation parsers can add their own, more complex, ones)

This is is the issue with oneOf https://github.com/joelittlejohn/jsonschema2pojo/issues/653 in the library we are using.

we are going to move schema validation to use spec schemas rather than the sdk ones, so these things i think won't be as big of an issue

@tsurdilo ok, whats exactly the plan? Currently the generated POJOS in the SDK already has the javax.annotations. The not generated ones does not.

the "non-generated" ones should be base classes only. maybe lets open a proposal for changes you want to do (open as issue if you want) and lets see what you have in mind and go from there?

Unfortunately, non generated ones are concrete classes, check here https://github.com/serverlessworkflow/sdk-java/tree/main/api/src/main/java/io/serverlessworkflow/api/workflow

@tsurdilo ok, whats exactly the plan? Currently the generated POJOS in the SDK already has the javax.annotations. The not generated ones does not.

I am not 100%. sure right now. Thinking about long term solution atm.


Feat/top view

basic implementation of top for nodes and pods
sometimes values are 0  and 0 for like a second and then they refresh well. we can address this later.
ready to merge.

Done!

sorting not working.
i'm on it

ok fixed sort, welcome to test

not accumulating multi container metrics

i honestly think it's ready now @Ramilito

Lets merge it! I'll create a issue to remove the informer part and add an with_informer option to the builder


How can i contribute ?

How can I contribute ?
@Wiki page is not loading and I couldn't find how to simulate Android development environment in my local machine.

some requests features are missing and need to port over into java.
https://github.com/mgp25/Instagram-API
You can use this as reference and port it over into java. some etiquette would be to keep code style same, organized, documented, etc. Open a pull request after!

Yes, there are many things that we are still not supporting. For Android check #5.
You can submit PRs, I'm looking at them more frequently than issues, due to lack of time available.


Single tags page

name: Feature request
about: Suggest an idea for this project

Feature request

What is the expected behavior?
a single page for tags ie something like a sidebar with listed active tags and on the side shows all the post with said tag that user selected
What is motivation or use case for adding/changing the behavior?
overall less files no two diff pages for tags when can just be on one single page
How should this be implemented in your opinion?
mentioned up there ^
Are you willing to work on this yourself?
yes

We already have a tags page which the number of blog posts using each tags. Can you please elaborate more on what do you want to improve specifically?

yes yes i get what you mean but there is technically 2 pages ie /src/pages/tags/index.js as well as /src/templates/tags.js which in what im saying is do this put the page in /src/templates/tags.js and have a sidebar on that page that list all available tags then on the right side have the actual post with said tag that is selected   small visual of what i mean
tags                                                      post with tag 1

tag1                                     post_link ie title of post
tag2                                                      post excerpt
tag3
and when u click lets say tag 3 or two the stuff on the side changes to fit that tag
better than having two diff pages when can just all be on one page

ill make a mockup and take a screen shot for a visual of what i mean give me a sec to make it

kinda like this except on the right side is a list of all available tags


Update crd.yaml to use apiextensions.k8s.io/v1

apiextensions.k8s.io/v1beta1 is deprecated: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#customresourcedefinition-v122
Fixes #53

This is also covered in #58

Closing PR in favour of #58.


Bug OCPBUGS-4629: Set controller's SyncPeriod to 1 hour

The Machine controller that we run doesn't set the SyncPeriod. This means that Machine objects are not periodically reconciled with the status of them in OpenStack.
This is a problem when the user adds a floating IP to a Server manually. In such case cloud-provider-openstack will notice it and update the Node object accordingly. This will cause Kubelet to generate a new CSR with updated SANs including new FIP address. Then cluster-machine-approver will compare the list of SANs from the CSR with Machine's Status.Addresses and will never find the FIP failing the approval.
This commit sets SyncPeriod of the controller to 1 hour in order to make sure Machines are periodically reconciled with the state of the instances in OpenStack just like Nodes are.

/jira refresh

/assign @mdbooth

@dulek Do we know how much work a full re-sync does? Is this going to kill PSI again?

@mdbooth: For a noop it seems to be 2 list-by-name queries - one done from Exists() and another from Update() (reconcile()). I don't think we could safely cache these results in MAPO and it's probably difficult to change declaration of Exists to return an object instead of just boolean.
Do you think that running these queries every 10 minutes is dangerous?

Yep, in terms of the actuator we've got a single call to Exists() and a single call to Update() per-machine, every 10 minutes.
In the stable case this will be:
Exists:

Keystone authentication
Nova list by name

Update:

Keystone authentication (again 🙄)
Nova get by ID

With a sync period of 10 minutes we will be doing this once every 6 seconds in a 100 node cluster, or about 1 API call every 1.5 seconds in the stable case. That seems like a lot just to detect out-of-band changes to machines we'd rather you didn't make in the first place.
IIUC the issue this is trying to address is:

A user manually adds a floating IP to a machine
The kubelet on that machine subsequently creates a CSR which includes the floating IP
The CSR is not automatically approved because it includes the manually added floating IP

In order to not break automatic CSR approval in this case, the fastest we have to re-sync is related to the amount of overlap there is in the CSR approval, i.e. how long can it take for the CSR to be approved before stuff breaks? We just need to be faster than that.
Another point is that if the user is manually reconfiguring their machine they could manually approve a CSR until the machine is updated.
I don't think we can set this by default to anything less than an hour. As the only thing it affects (that we're aware of) is CSR approval, if the CSR approval process allows us to make that longer we should make it longer. And ideally it would be configurable, because a reasonable setting for a 10 node cluster is likely not reasonable for a 300 node cluster. However, we can be pragmatic about that if the answer to the other question means we can make it a long time anyway.

Another idea might be to add a new event source to the machine controller manager which watches for unapproved CSRs older than X and schedules a reconcile for those machines specifically.
@JoelSpeed would 👆 be feasible/overkill?

Setting to an hour would probably help, CSRs are regenerated if they aren't approved within 15 minutes so you'd see up to 4/5 CSRs I would expect, but I don't think it would be the end of the world.

Another idea might be to add a new event source to the machine controller manager which watches for unapproved CSRs older than X and schedules a reconcile for those machines specifically.

This sounds like a nasty tight coupling IMO, but it's not impossible. What if we reversed that logic though, what if whenever a new CSR was created, the CSR approver annotated the Machine with the name of the CSR it linked and failed to approve, annotated, set a status condition, something that would be available on the Machine to say hey, we found a CSR but it's not yet approved.
When the CSR approver updates the Machine, it would trigger the Machine controller to re-reconcile, which would in turn, assuming it updates it's status, trigger the CSR to re-reconcile and be updated. As long as the condition added to the machine is stable, that shouldn't cause too much toil

/remove-lifecycle stale
@dulek Do you want to close this in favour of https://github.com/openshift/cluster-machine-approver/pull/183, or do you want to keep this open in case the other change doesn't happen?

The only downside of the other approach is that I'm not sure it's implementable in a manageable way when we support two types of Machine objects. Let's keep this one floating a bit more, I'll make sure to close it once I'm sure https://github.com/openshift/cluster-machine-approver/pull/183 works.

@mdbooth, @JoelSpeed: Can we just go with this approach for now? I hardly have time to try the other one and we probably want Machines to be reconciled periodically anyway.

@mdbooth, @JoelSpeed: Kind reminder.

/retest-required

/retest-required

/approve
/lgtm

/retest

Oh, this won't get in now, we need to wait for branching. Testing it doesn't hurt though.

/retest

/retest

/retest

/retest

/jira refresh
/label acknowledge-critical-fixes-only

/jira refresh

/cherry-pick release-4.14


docs: Use @rest-hooks/react directly in docs

Motivation

While rest-hooks package is still usable; we want new users to simply use @rest-hooks/react directly

Codecov Report

Merging #2283 (153451e) into master (0353703) will not change coverage.
 Consider uploading reports for the commit 79db6c9 to get more accurate results

@@           Coverage Diff           @@
##           master    #2283   +/-   ##
=======================================
  Coverage   98.30%   98.30%
=======================================
  Files         124      124
  Lines        1950     1950
  Branches      293      293
=======================================
  Hits         1917     1917
  Misses         16       16
  Partials       17       17

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Unhandled instruction: tlbp

Hi! I’m using an elf file from building the Banjo Kazooie US V1.0 decomp project, tried using the N64Recomp tool with the elf file - and getting this unhandled instruction error.
Also got unhandled mfc0 and mtc0 cop0 (EntryHi 10 value) but managed to implement the handling for it myself - to be fair I’m not sure if simply adding cop0_entryhi_write/read for mfc0 and mtc0 cases will work when trying to use the func files at a later stage, but it has at least allowed the N64Recomp program to progress further - as those were the first errors.
The function is boot___osProbeTLB.
If I understand this project correctly - can I simply write the C equivalent of this function myself, and provide it in a manual_funcs file which I then point to in the .toml file or am I on the wrong track here?
I have to admit I’m completely fresh in the decomp/recomping of N64 games scene but I got very curious about these things after finally getting a 100% run on the Majoras Mask recomp, 20 years after trying the game for the first time.

Heres a screenshot of the result of running readelf -s banjo.us.v10.elf | grep boot___osProbeTLB

objdump gives me this:
mips-linux-gnu-objdump -d banjo.us.v10.elf | grep -A20 80003bd0
`800030b4:       0c000ef4        jal     80003bd0 <boot___osProbeTLB>
800030b8:       8fa40018        lw      a0,24(sp)
800030bc:       8fbf0014        lw      ra,20(sp)
800030c0:       27bd0018        addiu   sp,sp,24
800030c4:       03e00008        jr      ra
800030c8:       00000000        nop
800030cc:       00000000        nop
800030d0 <boot___osSiDeviceBusy>:
800030d0:       3c0ea480        lui     t6,0xa480
800030d4:       8dc40018        lw      a0,24(t6)
800030d8:       27bdfff8        addiu   sp,sp,-8
800030dc:       308f0003        andi    t7,a0,0x3
800030e0:       11e00003        beqz    t7,800030f0 <boot___osSiDeviceBusy+0x20>
800030e4:       00000000        nop
800030e8:       10000002        b       800030f4 <boot___osSiDeviceBusy+0x24>
800030ec:       24020001        li      v0,1
800030f0:       00001025        move    v0,zero
800030f4:       03e00008        jr      ra
800030f8:       27bd0008        addiu   sp,sp,8
800030fc:       00000000        nop
80003bd0 <boot___osProbeTLB>:
80003bd0:       40085000        mfc0    t0,c0_entryhi
80003bd4:       310900ff        andi    t1,t0,0xff
80003bd8:       2401e000        li      at,-8192
80003bdc:       00815024        and     t2,a0,at
80003be0:       012a4825        or      t1,t1,t2
80003be4:       40895000        mtc0    t1,c0_entryhi
...
80003bf4:       42000008        tlbp
...
80003c00:       400b0000        mfc0    t3,c0_index
80003c04:       3c018000        lui     at,0x8000
80003c08:       01615824        and     t3,t3,at
80003c0c:       1560001a        bnez    t3,80003c78 <boot___osProbeTLB+0xa8>
80003c10:       00000000        nop
80003c14:       42000001        tlbr
...
80003c24:       400b2800        mfc0    t3,c0_pagemask
80003c28:       216b2000        addi    t3,t3,8192
80003c2c:       000b5842        srl     t3,t3,0x1
80003c30:       01646024        and     t4,t3,a0
80003c34:       15800004        bnez    t4,80003c48 <boot___osProbeTLB+0x78>`

The boot_ prefixed libultra functions in the BK decompilation need to be handled just like the normal libultra functions are. Each function either needs to be ignored (like this one) or reimplemented (like the boot raw PI DMA function). It's a quirk of the build system that BK decomp has due to needing two copies of libultra in it, which happen to be familiar with since I was the one who set it up. Unfortunately I don't have much free time at the moment, so I won't be able to go through the exact steps to deal with this problem.
With that said I'm already working on a BK recomp on the backburner; it may be a while before I get time to more actively work on it. Don't let that stop you if you're doing this for fun of course, but I figured it'd be good to let you know.

Ah ok, so they should all be ignored. Thanks.
However after ignoring them as they came up, this one appeared: func_8026A2E0
readelf gives me this:
9090: 8026a2e0  1348 FUNC    GLOBAL DEFAULT   12 func_8026A2E0
I don't suppose that one is also part of the functions that should be ignored? I haven't looked into its content yet but it's an mfc0 value exception 13.

As far as func_8026A824 goes - it was NOT due to anything related to mfc0 - it's actually this:
unable to find jump table for 'jr $s2' at 0x8026A8D0 in func_8026A824

func_8026A2E0 and func_8026A824 are libultra functions with missing names, they can be ignored since they're part of the thread scheduler which is reimplemented in the runtime.

Ok, thanks! Closing the issue.


Support for d3 4.x

What is the support for d3 4.x like? Any known issues?

Has anyone ran into issues with this?

Yep, I did.
Run stacked chart and faced d3.layout.stack() error because d3.layout got undefined. (read somewhere it was split into a separate file for d3 4.x)
Reverted back to latest 3.x branch and it solved my issue.

I'm interested about this, I'm new to d3.js and the examples talk about d3.layout and I can't find this file. Even if there is a "recent" 1.6 version of rickshaw, nothing looks updated on the website...

I wish a core contributor would respond, the lack of response kind of tells me this project is dead :(. I wish I was better at charting/visuals and I had more time, then I would contribute in the time in which I don't have :(

I'm comprehensively updating the code and examples to work with d3 v4; my team uses Rickshaw and we can't keep postponing the d3 version upgrade.  I'll push my code to GitHub this weekend, as soon as I clean up my commit history for public consumption.
Would anyone @shutterstock be interested in merging a PR to update to d3 v4?  I can see from #517 that y'all have largely stepped back from maintaining this.  I'm happy to send a PR with my code and to help keep the code healthy if you want contributions.  Or if you'd rather keep this repo stable, I'm also happy to keep my fork running for my team (and anyone else who would prefer to use that).

@butterflyhug any updates on this?

I've got code on my fork.  Note that my current default branch is d3v4 -- not master.
I've added a section at the top of the readme to note a couple minor issues that I noticed from testing my code with the provided examples.  I've also tested with the existing unit tests and my team's own codebase, both of which seem to be working flawlessly.  I'm sure that there are a few features and/or configurations that haven't been covered by any of these testing methods, and those things may be broken.
I'm still happy to send a PR with my changes if @shutterstock is interested.  In the meantime, I'm happy to take PRs to address remaining issues on my fork.  If we get to a point where there are no known issues and I still haven't heard any official interest in supporting D3 v4, then I'll think about our options for independently publishing releases and etc.

@butterflyhug:

Would anyone @shutterstock be interested in merging a PR to update to d3 v4? I can see from #517 that y'all have largely stepped back from maintaining this. I'm happy to send a PR with my code and to help keep the code healthy if you want contributions. Or if you'd rather keep this repo stable, I'm also happy to keep my fork running for my team (and anyone else who would prefer to use that)

Yes, I would be more than happy to look at a PR for this update, and I would love to have help maintaining this repository. PR away!

I'm still happy to send a PR with my changes if @shutterstock is interested.

:wave:

Any fix should include replacing deprecated calls to d3.scale.linear() with d3.scaleLinear()

What's missing here? Response from @shutterstock?

Updated title and issue to 5.x since we are now two majors behind.


Does the ingress controller support AKS with Kubenet (Simple Networking)?

We have an existing AKS cluster using Kubenet (Simple Networking)
The Brownfields https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/docs/setup/install-existing.md shows that you need advanced networking.
I would like to use AKS kubenet with an application-gateway-kubernetes-ingress.
Thx

AKS with Kubenet is dependent upon AppGw v2 SKU supporting UDR on the subnet.

Hello,
Any update for AppGateway + kubenet?
I wonder how the application gateway can ping pod which are in an isolated subnet

Hello,
Any update for AppGateway + kubenet?
I wonder how the application gateway can ping pod which are in an isolated subnet

@akshaysngupta Is it possible for you to gives us a status on the integration of App Gateway as ingress controller with Kubenet?

Yes, this is now supported. We will update the docs soon on this.
If you want to try this quickly, just associated the AKS route table in node resource group to the subnet used by Application Gateway.


daemon: fix conntrack map dump wrt addresses

The CT dump currently shows swapped src/dst address entries even
though it's correctly using src address resp. dst address as data.
Issue is that 7afe903203c3 ("bpf: Global 5 tuple conntrack.") did
not swap the initial tuple for the lookup when converting from
local to global table, and all the current code right now is doing
workarounds in order to not break CT table during version upgrade.
Thus same needs to be done here for the dump. Issue became more
apparent after aaf6ba39ad4e ("ctmap: Fix order of CtKey{4,6} struct
fields"), which might have had been swapped on purpose but without
further comments in the code on why it was swapped on daemon side.
In this case, reverting aaf6ba39ad4e doesn't fully fix it either
since then direction also needs to be swapped. Instead, make it
less confusing and only swap what needs to be swapped, that is, the
address parts since in the datapath this is the only thing that
should have been done but was missed back then. For next major
version upgrade (aka 2.0), this will be properly fixed (at the
cost of disruptive upgrade).
Fixes: 7afe903203c3 ("bpf: Global 5 tuple conntrack.")
Fixes: aaf6ba39ad4e ("ctmap: Fix order of CtKey{4,6} struct fields")
Signed-off-by: Daniel Borkmann<EMAIL_ADDRESS>

This change is

test-me-please

Coverage remained the same at ?% when pulling 742efdfc8c4f34d9806a44ca9302e7abe24df09c on pr/ct-dump-fix into adedcff32759af08c8d20a5d1e891ae01e33f9b3 on master.


Build issues on x86_64

I would like to run redtail on my GTX1070 laptop for controlling a small FPV vehicle remotely.
While trying to build redtail on x86_64, I'm getting cmake errors related to Qt5 such as the example below.  I already have the Qt5 packages on my system.  What am I missing here?  What other info can I provide to help solve this issue? install log and dpkg -l output are attached
CMake Error at /usr/share/cmake-3.5/Modules/FindCUDA.cmake:1693 (add_executable):
Target "caffe_ros_node" links to target "Qt5::Core" but the target was not
found.  Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
Call Stack (most recent call first):
caffe_ros/CMakeLists.txt:124 (cuda_add_executable)
log_121217.txt
dpkgList.txt

It seems like you are running our Jetson install script (jetson_ros_install.sh) on your x64 machine directly. We have not tested this scenario and it's not supported - the script is intended to be used only with Jetson/JetPack. I can suggest 2 options:

Use our Docker image which has all the components required to run redtail. Once you build and run the container, you can set up the ROS nodes in any way you like, for example, run the controller node on your vehicle, while DNNs - on your laptop (from Docker).
Try fixing the script. When you do that, I strongly recommend doing so from a Docker container to avoid polluting your host system. You don't have to use our image, but you can use it as a starting point and skip the components you don't need (e.g. Gazebo etc).

Thanks you for the suggestions.  Either of those should work for me.  The Docker image solution seems fast and easy, except that it appears to be a 6.21GB file, and unfortunately I'm limited by a slow rural internet connection.
For installing directly on my machine, it would help if you could suggest what dependencies are required to build the redtail ros nodes?  Other than what the script installed, I built opencv2.4.13 from source after the script complained about wrong opencv version.
I tried manually building caffe_ros_node in the workspace setup by jetson_ros_install.sh
I understand from the output that it's a linker error, but I'm confused how to solve it.  I do have qt5 on my machine.  I.E. qgroundcontrol builds from source without any errors.
chris@GL702VSK:~/ws$ catkin_make caffe_ros_node
Base path: /home/chris/ws
Source space: /home/chris/ws/src
Build space: /home/chris/ws/build
Devel space: /home/chris/ws/devel
Install space: /home/chris/ws/install

Running command: "make cmake_check_build_system" in "/home/chris/ws/build"

Running command: "make caffe_ros_node -j8 -l8" in "/home/chris/ws/build"

[ 25%] Linking CXX executable /home/chris/ws/devel/lib/caffe_ros/caffe_ros_node
/usr/bin/ld: cannot find -lQt5::Core
/usr/bin/ld: cannot find -lQt5::Gui
/usr/bin/ld: cannot find -lQt5::Widgets
/usr/bin/ld: cannot find -lQt5::Test
/usr/bin/ld: cannot find -lQt5::Concurrent
/usr/bin/ld: cannot find -lQt5::OpenGL
collect2: error: ld returned 1 exit status
caffe_ros/CMakeFiles/caffe_ros_node.dir/build.make:203: recipe for target '/home/chris/ws/devel/lib/caffe_ros/caffe_ros_node' failed
make[3]: *** [/home/chris/ws/devel/lib/caffe_ros/caffe_ros_node] Error 1
CMakeFiles/Makefile2:897: recipe for target 'caffe_ros/CMakeFiles/caffe_ros_node.dir/all' failed
make[2]: *** [caffe_ros/CMakeFiles/caffe_ros_node.dir/all] Error 2
CMakeFiles/Makefile2:909: recipe for target 'caffe_ros/CMakeFiles/caffe_ros_node.dir/rule' failed
make[1]: *** [caffe_ros/CMakeFiles/caffe_ros_node.dir/rule] Error 2
Makefile:485: recipe for target 'caffe_ros_node' failed
make: *** [caffe_ros_node] Error 2
Invoking "make caffe_ros_node -j8 -l8" failed

My guess would be OpenCV versions issue. caffe_ros by itself does not use any qt libraries so it is most likely coming from OpenCV. Having multiple versions of OpenCV is a pain (to put it mildly) so this where Docker might help. Another idea would be to build OpenCV without QT (and other redundant in this case components, like samples, tests etc) and see if it helps.
The main reason our code uses version 2.4.13 is that this is the version installed by JetPack so we wanted to avoid building OpenCV in our scripts.

I was able to build caffe_ros after rebuilding OpenCV 2.4.13 without QT.  The docker image is still downloading...
However, now there is an issue building caffe_ros_tests that is giving me troubles.
tests.cpp:(.text+0x42): undefined reference to `cv::imread
Log is pasted below.
Searching suggested that missing opencv_imgcodecs could be the issue, but it seems to be on my system.
I have verified that opencv_imgcodecs.so is present on my LD_LIBRARY_PATH
chris@GL702VSK:~/ws$ echo $LD_LIBRARY_PATH
/usr/local/lib:/home/chris/mkdev/opencv/build/lib:/home/chris/ws/devel/lib:/opt/ros/kinetic/lib:/home/chris/Qt/5.9.1/gcc_64/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/targets/x86_64-linux/lib/:/usr/lib/x86_64-linux-gnu/gazebo-7/plugins
chris@GL702VSK:~/ws$ locate opencv_imgcodecs.so
/home/chris/mkdev/opencv/build/lib/libopencv_imgcodecs.so
/home/chris/mkdev/opencv/build/lib/libopencv_imgcodecs.so.3.3
/home/chris/mkdev/opencv/build/lib/libopencv_imgcodecs.so.3.3.0
/usr/local/lib/libopencv_imgcodecs.so
/usr/local/lib/libopencv_imgcodecs.so.3.3
/usr/local/lib/libopencv_imgcodecs.so.3.3.0
chris@GL702VSK:~/ws$ catkin_make caffe_ros_tests
Base path: /home/chris/ws
Source space: /home/chris/ws/src
Build space: /home/chris/ws/build
Devel space: /home/chris/ws/devel
Install space: /home/chris/ws/install

Running command: "make cmake_check_build_system" in "/home/chris/ws/build"

Running command: "make caffe_ros_tests -j8 -l8" in "/home/chris/ws/build"

[ 25%] Building CXX object gtest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 50%] Linking CXX shared library libgtest.so
[ 50%] Built target gtest
[ 75%] Building CXX object caffe_ros/CMakeFiles/caffe_ros_tests.dir/tests/tests.cpp.o
[100%] Linking CXX executable /home/chris/ws/devel/lib/caffe_ros/caffe_ros_tests
CMakeFiles/caffe_ros_tests.dir/tests/tests.cpp.o: In function readImage(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)': tests.cpp:(.text+0x42): undefined reference to cv::imread(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int)'
collect2: error: ld returned 1 exit status
caffe_ros/CMakeFiles/caffe_ros_tests.dir/build.make:117: recipe for target '/home/chris/ws/devel/lib/caffe_ros/caffe_ros_tests' failed
make[3]: *** [/home/chris/ws/devel/lib/caffe_ros/caffe_ros_tests] Error 1
CMakeFiles/Makefile2:934: recipe for target 'caffe_ros/CMakeFiles/caffe_ros_tests.dir/all' failed
make[2]: *** [caffe_ros/CMakeFiles/caffe_ros_tests.dir/all] Error 2
CMakeFiles/Makefile2:941: recipe for target 'caffe_ros/CMakeFiles/caffe_ros_tests.dir/rule' failed
make[1]: *** [caffe_ros/CMakeFiles/caffe_ros_tests.dir/rule] Error 2
Makefile:498: recipe for target 'caffe_ros_tests' failed
make: *** [caffe_ros_tests] Error 2
Invoking "make caffe_ros_tests -j8 -l8" failed

It looks like it's the wrong version of libopencv_imgcodecs: in your output, it's 3.3, not 2.4.

Awesome, nice work Alexey! I was able to get caffe_ros_node running on my x86_64 laptop by building with your alexeyk/2.0 branch.  Now I've got the tx1 running trailnet onboard the rover, and the yolo node running on my gtx1070 laptop.


Upgrade pgrx to 11.2

You would need to update CI files too IIRC.

Indeed, I found a pgrx version reference in the Dockerfile and a github workflow. Fixed and pushed.

👋 I'm hitting a similar version issue when running pgrx 0.11.3. Could I make a PR to relax the version checks here (~0.11, meaning, >=0.11.0, <1.0.0) ?

Could I make a PR to relax the version checks here (~0.11, meaning, >=0.11.0, <1.0.0) ?

That would actually mean >=0.11.0,<0.12.0
If you set it to that in the repo and it works with 0.11.3, I don't mind merging it.


Use of different Icons aside from Core UI Icons

Hi,
Is there any possibilities that we can use our own Icons? or we can use another Icons aside from Core UI?
and if so, is there documentations on how to integrate it?
Hope to hear from you Core UI team.
Thanks in advance.

As extension to this issue, I'm missing the documentation on how to use FontAwesome icons in the Sidebar component of coreUI later than version 5.
All issues and stackoverflow posts regarding this are linked to older versions of CoreUI.
I tried extending the IconComponent object and overriding several methods like getInnerHTML and others related to DOM generation - But I haven't succeeded yet.
I also tried putting a FontAwesome IconComponent object on the fitting object position in the Menu structure - This also didn't work.
Looking forward to hearing from you
Jan Nox
Maybe there's another way?

I have the same question. It looks like that it is impossible to use other icons than the CoreUI icons at the moment.
I did not found a way to overwrite the default template, without copy&past all the navigation related files
https://github.com/coreui/coreui-angular/blob/40f7429aab634d19e97c7072c4b7e2b5f1cbf576/projects/coreui-angular/src/lib/sidebar/sidebar-nav/sidebar-nav-group.component.html#L22-L39
I'm using Coreui 5

Same question here, the CoreUI Icons are not enough, I need some other icons from FontAwesome. How can I use them?

You can use FontAwesome CSS classes only at the moment, we're planning to introduce svg support later this year.
for example:
use package manager to install FontAwesome
npm install --save @fortawesome/fontawesome-free

import FontAwesome CSS in your src/scss/styles.scss file:
@import<EMAIL_ADDRESS>
use FontAwesome CSS classes in your src/app/layout/default-layout/_nav.ts configuration file
export const navItems: INavData[] = [
  {
    name: 'Dashboard',
    url: '/dashboard',
    icon: 'fa-sharp fa-solid fa-home',
  },
...

See also: https://docs.fontawesome.com/web/setup/get-started


Investigate behaviour and styling of Readonly Input Elements

Bug Report
Description

As of now there are no defined standards for the behaviour/styling of " Readonly Input Elements" when defined using the html readonly attribute.
Expected Behavior

The readonly behaviour and styling for input elements should be consistent throughout all terra components.
@ Mentions

@neilpfeiffer @jeremyfuksa

We will need to spike on this from a UX standpoint before we are ready to start working on this.


fix: SCORM attachment link error

Checklist

[x] the contributor license agreement is signed
[x] commit message follows commit guidelines
[ ] tests are included
[x] screenshots are included showing significant UI changes
[ ] documentation is changed or added

Description of change
Fix missing attachment links for unzip files.
The issue is in staging folder all unzip files are not transferred properly and still stay in the folder _uploads which will be removed before moving to institution folder.
#3465
Before:
Upload a zip file:

Unzip:

Save:

After fix
Upload zip file:

Unzip:

Save:

I imagine two things:

This used to work, and hence why it was raised as a bug only recently; and
This probably came about as a result of the rework of the upload control.

With that in mind, perhaps we should look back to how it used to be done, starting with the following references to when the change was made:

This issue: #267 (which unfortunately lacks and 'why' for such a large overhaul - may have been the desire to move more code to scala and purescript)
This PR: #291
This final commit into master: 20ee6ceb6a55d5197ded4d49f41332033ead33fd

Keep in mind that since those the purscript code was replaced with typescript and the scala updated as needed.

I found this bug is first mentioned in 2018 but I was confused since it says it was fixed. https://github.com/openequella/openEQUELLA/issues/576

Could it be because that change was not merged to develop?  I can't find that commit history in develop branch locally.

Could it be because that change was not merged to develop? I can't find that commit history in develop branch locally.

If you open any file involved in that commit, can you see the changes ?

Could it be because that change was not merged to develop? I can't find that commit history in develop branch locally.

You can also try to cherry pick that commit in your working branch and see if it can fix the issue.

That commit is not a merge commit. I think it was pushed to stable-6.6 directly.

Work on this continues elsewhere, closing PR.


[Access Request] report 5705 "Transfer Receipt"

Describe the request
Could you please change the access scope of the variables TransferFromAddr and TransferToAddr to protected?
   protected var
    TransferFromAddr: array[8] of Text[100];
    TransferToAddr: array[8] of Text[100];

Additional context
We sometimes need to changes this in ReportExtensions for PTEsInternal work item: AB#492876

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 14454.


Text for new user and help

Implement english intro text for new user. It is very important, as the first experience is MOST valuable
Also - we need to make /help texts.
This is one task, because these text should be mostly identical

Help done, but first experience is not so good
May be add 20 words at the start?


Add static endpoint override to test/spoof

When testing locally its common to use a cluster which both does not
have resolvable dns for ksvcs and does not have public IPs. Support
overriding the endpoint for knative's ingress controller to a static
value which can be set to e.g. the knative-ingressgateway nodeport.

Looks like this is failing the MD lint checker and is blocking https://github.com/knative/serving/issues/2885.
Anything to move this along?

Yeah ive seen this several other places, I think we added a dead link checker to test-infra which only runs when md files are changed so it didnt fail when being added since that PR wasnt touching any md files. Now whenever we touch an md file we fail due to already existing dead links. IMO we should back that change out and always run the dead link checker so we avoid creating this case again.

/retest

/test pull-knative-pkg-build-tests

Once https://github.com/knative/pkg/pull/265 is merged, the build tests will pass (just /retest). The new link checker deals better with URLs and understands references to local files.

/lgtm
/retest
/hold cancel


🍒 Cherry pick PR #8940 to staging 🍒

🍒 Cherry pick https://github.com/Expensify/App/pull/8940 to staging 🍒

🚀 Cherry-picked to staging by @roryabraham in version: 1.1.57-13 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by @chiragsalian in version: 1.1.57-17 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[Feature request] Gradient background in image editor

Platforms
Android, iOS
Description
Can we add background gradient to selected image?. i used this to add color to background.
imageEditorTheme: const ImageEditorTheme(           background: Color.fromARGB(255, 54, 58, 130),           loadingDialogTheme:               LoadingDialogTheme(textColor: Color.fromARGB(255, 225, 255, 200)),         ),
But it can take only one color, can we have gradient background property?
Why
Like instagram, we can get color from image and then make a gradient background.

To set a gradient background for a captured image, a small adjustment is required. By default, the editor captures only the image area and ignores the background. As a result, depending on the output format, the background will either be black or transparent.
However, if you wish to use a gradient as the background, you can refer to the "movable-background-example." In this example, you will need to update this section with the code provided below.
final canvas = Canvas(
  recorder,
  Rect.fromLTWH(0, 0, width.toDouble(), height.toDouble()),
);

// Define the gradient
const gradient = LinearGradient(
  colors: [Colors.blue, Colors.purple],
  begin: Alignment.topLeft,
  end: Alignment.bottomRight,
);
final paint = Paint()
  ..shader = gradient.createShader(
    Rect.fromLTWH(0, 0, width.toDouble(), height.toDouble()),
  );
canvas.drawRect(
  Rect.fromLTWH(0.0, 0.0, width.toDouble(), height.toDouble()),
  paint,
);

final picture = recorder.endRecording();


env.out.print  Resulting in skyrocketing memory

use "collections"

actor Main
  new create(env: Env) =>
    for i in Range(0, 10000000) do
      env.out.print(i.string())
    end

linuxmint18 64
llvm3.6
0.2.1-966-g6e98a13 [debug]

garbage collection only happens between behaviors so, all those strings you are creating won't get garbage collected until you are done iterating over the range.
There's a tradeoff here. My running gc only between behavior calls on an actor, you get predictable performance within the behavior and an easier to write, modify and understand garbage collector.
OTOH, you can hang yourself by doing what you are doing.
You also are suffering from a producer, consumer problem here. the loop is going to produce much faster than it can be written to stdout, which is going to result in a lot of backup.
Note, I'm not saying there might not be something else going on here as well, but you are definitely triggering a couple things that will result in a good amount of memory usage.
If you try running your program with --ponygcfactor=1 you will see a large difference in both memory usage and performance. The default for --ponygcfactor is 2. It means, each time we gc, double the amount of memory needed to trigger the next gc run. By setting it to 1, you sacrifice performance (gc more often) for memory usage.
On the producer/consumer front, we have had some discussions about how to do back pressure. Back pressure is a tricky topic at the runtime level though as what is right for one application isn't right for another.
Here is a version that uses much less memory:
use "collections"

actor Main
  let _foo: Foo = Foo
  var _sent: USize = 0

  new create(env: Env) =>
    go()

  be go() =>
    if _sent <= 10000000 then
      other(_sent)
      _sent = _sent + 1
    end

  be other(i: USize) =>
    _foo.print(i.string())
    go()

actor Foo
  be print(s: String) =>
    None

Note that we aren't printing. This is to show the impact of no producer/consumer issue.
Also note, this version is going to run slower because we will do garbage collection on all
those strings that are being generated.
Here's another version that can exhibit the producer/consumer problem:
use "collections"

actor Main
  let _out: StdStream
  var _sent: USize = 0

  new create(env: Env) =>
    _out = env.out
    go()

  be go() =>
    if _sent <= 10000000 then
      other(_sent)
      _sent = _sent + 1
    end

  be other(i: USize) =>
    _out.print(i.string())
    go()

If you run this and write it to your terminal, the the standard output actor is going to be much slower than the producer for it and memory usage will climb quickly. If you redirect output to /dev/null, the program will go much quicker in terms of outputting the strings and you will see much lower memory usage.

@jemc @sylvanc i think we need more documentation around this. certainly there is a pony pattern or two in here. Does performance deserve a section in the tutorial? I'm not sure on that. In my mind, the tutorial is "Pony for the Impatient" to get you started. And performance etc type things would be either patterns or part of another book specifically addressing performance or other more advanced Pony usages. That said, beginners will run into this so... ¯_(ツ)_/¯. Thoughts?

I think we should start with a Pony pattern or two - perhaps a "performance" section of the patterns collection.  We could also consider adding a "where to go from here" page at the end of the tutorial which links to the patterns collection.


Fix (#2985)

/retest

This is weird the fix is referencing a very old PR: https://github.com/knative-extensions/eventing-kafka-broker/pull/2985

OCF Webhook is merging this PR


Distance labels for env. distances of a ligand don't get cleared

This occurs for any given ligand, simply by turning on and off the env. distances.

Should be fixed in bd0204501640a178e50d2a207d152f447fbd66b0


dashboard for realtime data visualization

In order to calibrate the openbci device (i.e. make sure each electrode is connected, impedance is low), we need to stream data from the board to the 'dashboard' page of the app.  The basic setup should include:

realtime data stream of all channels
topographic head plot
fourier plot
'good connection' indication for each channel
someway to display impedance.
very basic filtering options

Ideally, this part of the app would be modular and customizable so that a programmer could easily write a custom 'plugin' and add it to the dashboard with little effort.  The components listed above would come in a default layout, but that layout should be mutable (i.e. the size/position of each component should be flexible).

Should it look similar to this? @pushtheworldllc and I are going to try and get the npm module hooked in, so this will be how we prove it's working

Yep, in terms of the structure.  I'd like to make it a little prettier than this.  Let's make a list tonight of the functionality we need.  It would be awesome if these windows could be movable and resizeable so users could customize their layout.  It should also be easy to add custom components to this if we want to make additions in the future

That'll be great. Once we have a list of everything we want this page to do we can ask @brennbanta and maybe get some good design in there. They will each be angular directives, and there is a library called ui-sortable, that extends jQuery's sortable library for angular. I used it in another project.

I see that ui-sortable sorts lists very nicely.  Do you know how it handles grids or multidimensional lists?  The sorting functionality used in keep.google.com is kinda what I am thinking about, except with resizeable windows

here's another option that may work as well:
http://rubaxa.github.io/Sortable/

full resizable windows I believe will be difficult and add a lot of complexity. What I think would be a better strategy would be to define a set of layouts, and allow users to drag the whole windows around to replace which one is in each cell. So it is still customizable, but easy to program and lightens the page up. We can then plan the look and feel for those specific sizes, making sure each module is usable at the predetermined sizes.

ng-sortable is anther option (https://github.com/a5hik/ng-sortable it is the angular version of sortable), might be a good idea. We'll compare the two, but I like not having to use jQuery :)

sounds good.  my one hold back with a template approach is that is the user wants to write a custom directive for a new visualization, it won't fit nicely into the scheme....maybe having a more complicated set of templates, or a lattice 'snap' structure would solve the problem

another possibility is that custom visualizations could pop open a new window..but that might be more trouble than its worth...just an idea :)

What I'm imagining for a template scheme is merely zoned sections for each visualizations. So for template for having only 1 graph show up, two split, three with one being large and the other two stacked...etc. We can always instruct people to build custom visualizations that fit into those guidelines.

got it..and within each template, the graphs will be sortable or no? i.e. in the openBCI GUI above, you could drag the timeseries plot from the left to the right?

another advantage is that we wouldn't have to support custom visualizations, because that would all be handled through lightning

another advantage is that we wouldn't have to support custom visualizations, because that would all be handled through lightning

Made some progress on this front.  Its SUPER easy to get streaming data into the browser using this package. See below for some code that simulates 8 chans of sinusoidal data:
var Lightning = require('lightning.js');

var lightning = new Lightning();

lightning.lineStreaming([[0],[0],[0],[0],[0],[0],[0],[0]])
    .then(function(viz) {
      var counter  = 0;
      var data = [[0],[0],[0],[0],[0],[0],[0],[0]];
        setInterval(function() {
          for(i=0; i<8; i++){
          data[i][0] = (Math.sin( counter ) / 2) + Math.random() + i;
          counter += .1
          }
            viz.appendData(data); // appends to existing data
            // console.log("appended")
            // or
            // viz.updateData([Math.random()]); // replaces existing data
        }, 50);
    });

This opens up a server that serves the data/visualization, which you can then place into HTML using iframes or the pym.js npm package

and here's a nice tutorial on creating custom visualizations
https://www.youtube.com/watch?time_continue=832&v=3kZxNGj15-s

Boom.  streaming data in the browser from the simulator.
var Lightning = require('lightning.js');
var lightning = new Lightning();

var Board = require('openbci-sdk')
var board = new Board.OpenBCIBoard();

board.simulatorStart().then(function() {
  lightning.lineStreaming([[0],[0],[0],[0],[0],[0],[0],[0]])
      .then(function(viz) {
        var data=[[0],[0],[0],[0],[0],[0],[0],[0]];
        board.on('sample',function(sample) {
        //         /** Work with sample */
          console.log(data)
          for(i=0; i<8; i++){
          data[i][0]= sample.channelData[i+1];
        };
          viz.appendData(data);
          //console.log(sample.channelData[1]);
        });
    });
  });

have to be running a lightning server at ---> localhost:3000
@tashoecraft @teonlamont @pushtheworldllc

Score! I should be a way to change the rate the data comes out at! Attach a pic!!

AJ Keller
Founder & CEO
Push The World LLC
Work:<PHONE_NUMBER>
Cell:<PHONE_NUMBER>
Warning: The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and destroy this material.

On Jan 2, 2016, at 10:35 PM, Andy Heusser<EMAIL_ADDRESS>wrote:
Boom. streaming data in the browser from the simulator.
var Lightning = require('lightning.js');
var lightning = new Lightning();
var Board = require('openbci-sdk')
var board = new Board.OpenBCIBoard();
board.simulatorStart().then(function() {
lightning.lineStreaming([[0],[0],[0],[0],[0],[0],[0],[0]])
.then(function(viz) {
var data=[[0],[0],[0],[0],[0],[0],[0],[0]];
board.on('sample',function(sample) {
//         /** Work with sample */
console.log(data)
for(i=0; i<8; i++){
data[i][0]= sample.channelData[i+1];
};
viz.appendData(data);
//console.log(sample.channelData[1]);
});
});
});
have to be running a lightning server at ---> localhost:3000
@tashoecraft @teonlamont @pushtheworldllc
—
Reply to this email directly or view it on GitHub.

this is the default viz, we can customize it i.e. line width color etc

wow! :) :+1:

My only fear would be getting caught in a library that isn't widely used. There isn't nearly as many resources available for lightning as compared to d3. Checkout d3 here: http://d3js.org. It's by far the leading library used for visualizations. It does have more of a learning curve, but it is powerful enough to accomplish anything we want to do.
I don't quite understand the custom visualizations portion, as any type of graphing library should handle the custom portions that could give problems, and there will be other examples people can take on how t make their own.

I definitely sympathize with that.  However, lightning is pretty new and growing quickly. Also, It wouldn't replace D3 as a graphics library, they support d3 and other viz packages.  Basically, they are providing a visualization server to connect to and interact with via an API (write now they have clients for javascript, python, scala and R).  I think it would cut out a bunch of work to just have OpenEXP running a lightning server, which handles all the real-time data, and any future graphics we'd like to implement.
Here's their blurb about it:
Lightning is a data-visualization server providing API-based access to reproducible, web-based, interactive visualizations. It includes a core set of visualization types, but is built for extendability and customization. Lightning supports modern libraries like d3.js, three.js, and leaflet, and is designed for interactivity over large data sets and continuously updating data streams.

for example, here is literally all the code i needed to get realtime streaming data into OpenEXP:
<iframe src="http://localhost:3000/visualizations/ccd1bd8b-1ae4-43fe-82a5-5b527e4e098b/iframe/" width="600" height="600" >

Do you know how to get the server running without having to install it globally? I'm trying to think of the startup process, but they either say install node locally and run the server, or run off heroku/docker (which would require an internet connection). I installed it into the node modules and I may be able to just run it off there, it's just a really big node module, it doubles the size of the app with this one install.

i wasn't successful at getting it to run from the npm module, but i didn't futz with it too much.  I downloaded the standalone app and hosted that way.  for OpenEXP, if we do end up using lightning, we would want the node module going so it doesn't require internet.
do you think the app size is that critical? maybe we could strip down the module if that is an issue.
also, im open to doing the viz other ways, this way just seems flexible and very easy to implement

It's just a very large library, app size isn't critical, but it shouldn't be ignored. It does seem like a pretty solid library that would help with the constant reloading. I'll try and explore it and see if I can get it working and also look at the alternatives to try and make a more educated guess.

sounds good! let me know if I can be of some help, otherwise, I'll do the same

I talked to the lightning creator on their gitter:
andyh616 14:20
@mathisonian or anybody else in the gitter, I was hoping to get a sense for what flexibility / utility would be gained by going with a lightning server as opposed to writing data directly to a DOM with a D3 visualization or something like that. I'm building an open-source app for EEG visualization / running psych experiments and our team is debating whether to go directly from our node server to the app, vs. initializing a lightning server via node, and sending the data through that to the app. I like the idea of going through a lightning server because connecting the data to the visualization in the browser is super easy with an iframe/pym.js. Also, the viz could be shared publicly if desired. Are there other features I am not considering? Sorry for the convoluted question, I'm just trying to get the full scope of the package and see whether it makes sense for us
to use
mathisonian 14:22
the advantages i see are

no hassle with DOM manipulation / management of piping data into the browser
its easy to put new (potentially external) data into the visualization at any time, especially good if this is a generic EEG visualization
other people can use it
it may be the case that you just want to go straight to the browser
though I would note that its straightforward to use a “lightning” visualization straight in the DOM
so if you build it with the lightning spec then it could go either way

I would question about it being used in an offline electron app.

Hey all!
Very cool project. If you do end up using lightning, I'm happy to help guide and figure things out like integrating in the most efficient way. Excited to see where this goes!

@mathisonian What would be a good process of starting up the server when used inside of an electron app that needs to be able to be run offline. So preferably it will fire up the lightning server each time the app is launched.

@tashoecraft, re the offline app.  I think the app should be able to run seamlessly offline, but shouldn't be limited to that.  It would be awesome if we had the flexibility to port in experiments from the internet, connect with remote databases, or share visualizations with others.

@andyh616 I know, just the majority of features need to be able to handle offline use, like visualizations. I have to imagine the number of people who will be forced to be offline entirely will be small, it's just a good goal.

@tashoecraft you could either require('lightning-server') directly in your app, or have it spawn another process that ran the executable (node_modules/lightning-server/bin/lightning-server.js if you install locally).
We distribute as an electron app too, so it shouldn't be a there (although compiling sqlite can be a pain)

An update here:

[ ] viz is being worked on by @tashoecraft in D3
[ ] @andyh616 is going to work on making it lightning-friendly


[BP-1.16][hotfix] Backport FLINK-30378 and FLINK-31758 to fix incorrect doc

Backport FLINK-30378 and FLINK-31758 to fix incorrect doc

Thanks @TanYuxin-tyx, merging...


Ring Client support

I would like to have a ring client support similar to go-redis ring.
It basically enables you to use shards to scale out redis.
There are a couple of feature that we would need in this ring client, that is likely good to know in advance:

In our case we use it for rate limits at the ingress layer, so we have a lot operations per second that will exceed a single redis process and require redis shards.

One key feature for us would be also to be able to update the shards like in https://github.com/redis/go-redis/blob/b64d9deef330a51cfbd3e0425b6c26b27c1ee370/ring.go#L538-L540 (https://github.com/redis/go-redis/commit/6f7f800107ba67310cd822d35b4558255f702ff1).

In some cases we need even to built a vnode layer on top to spread operations to more shards https://github.com/zalando/skipper/blob/master/net/redisclient.go#L218 , so it would be great if we could have an option to set also the ConsistentHash function to be used. In general rendezvous hash was the best option in my experience.

Hi @szuecs,
We currently don't support the ring sharding because we already support the standard Cluster. I actually don't understand why to use the ring sharding over the standard Cluster. But I will still be grateful to accept a PR of ring client implementation.

The problem is that with cluster you create a safe setup but not a scalable one. We need a non safe scalable system. https://redis.uptrace.dev/guide/ring.html#keys-distribution
A ring client does a rendezvous hash (much better than crc16 modulo N) of the key and every redis is just a single store and has no overhead to do master/slave like behavior. The ring has no problems like partitioning or complete downtime if half the nodes are down.

has no overhead to do master/slave like behavior.

The standard cluster doesn't need to have replicas either. You can have a cluster that consists of only primary nodes.

The ring has no problems like partitioning or complete downtime if half the nodes are down.

That is an infrastructure failure that users should take very seriously. Normally, no one would like to run applications on such an unstable infrastructure. I believe even the users of the ring don't want to lose half of their nodes. So treating the ability to survive in such a case as a selling point seems unwise to me.
But, since the ring is so simple, I am not against having a ring implementation in this library. Here are some tips if someone wants to implement it:

The ring implementation can be a slim layer on top of multiple valkey.Clients. This simplifies the implementation:

ring := valkey.NewRing(valkey.RingOption{
    Shards: map[string]valkey.Client{
        "shard1": client1,
        "shard2": client2,
    }
})

The implementation can use the cmd.Slot() as part of the rendezvous hash:

cmd := client.B().Get().Key(key).Build()
cmd.Slot()

I think you mean all the Slot() in github.com/valkey-io/valkey-go/internal/cmds/cmds.go , right?

Yes, the value returned from Slot() functions is the hash used by the standard cluster.

Maybe I should create separate issues for autoscaling feature and vnode hashing feature. What do you think about this?

No need. We can address this in this issue. It looks like we need an interface to pick a shard, for example:
type RingShardPicker func(cmd valkey.Completed, activeShards []string) string

What do you think about func (*Ring) SetAddrs(addrs map[string]string) to implement the feature I asked to implement 2.?

No blocker, but I would prefer this one:
func (*Ring) AddShard(shard string, client valkey.Client)
func (*Ring) RemoveShard(shard string)

The reason is that I want to keep the implementation simple and avoid constructing valkey.Client in the implementation.

We don't need to use the SetSlot(key) because the crc16 value should already be calculated during the command construction:
cmd := client.B().Get().Key(key).Build() // the crc16 is calculated here.
cmd.Slot() // we just need to retrieve it by using Slot().

I think we could implement the rendezvous hash like this:
func RendezvousRingShardPicker(cmd valkey.Completed, activeShards []string) string {
    slot := cmd.Slot()
    sort.Slice(activeShards, func(i, j int) bool {
        return (slot ^ hash(activeShards[i])) < (slot ^ hash(activeShards[j]))
    })
    return activeShards[0]
}

I just reconsidered the RingShardPicker I proposed, and I found it left no space for performance optimization. The above RendezvousRingShardPicker will work but its performance is not acceptable.
To have spaces for performance optimization, we might still need an interface like this:
type RingShardPicker interface {
    AddShard(shard string)
    RemoveShard(shard string)
    Pick(cmd valkey.Completed) (shard string)
}

This interface allows implementations to have their optimized structure for faster lookup.

We don't need to use the SetSlot(key) because the crc16 value should already be calculated during the command construction:
cmd := client.B().Get().Key(key).Build() // the crc16 is calculated here.
cmd.Slot() // we just need to retrieve it by using Slot().

My thinking was more to replace crc16 by rendezvous. As far as I know double hashing is not very good, because it won't have the properties of the one hash function that you want to use.
I think it would be something like:
cmd := ringClient.B().Get().Key(key).Build() // rendezvous(key) is calculated here, maybe via Pick() from RingShardPicker
cmd.Slot() // does not change as before use the cached value

I just reconsidered the RingShardPicker I proposed, and I found it left no space for performance optimization. The above RendezvousRingShardPicker will work but its performance is not acceptable.
To have spaces for performance optimization, we might still need an interface like this:
type RingShardPicker interface {
    AddShard(shard string)
    DelShard(shard string)
    Pick(cmd valkey.Completed) (shard string)
}

This interface allows implementations to have their optimized structure for faster lookup.

The interface seems to make a lot of sense, thanks!
I would think that a ringClient would implement similar to Client, but instead of crc16 it uses rendezvous and ringClient also implements  RingShardPicker (at least for AddShard/DelShard).


Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)

[ ] iOS
[x] Android
[ ] UWP
[ ] .NET Standard

Feature Library

[ ] Core
[x] BluetoothLE
[ ] Locations
[ ] HTTP Transfers
[ ] Notifications
[ ] Sensors

Version of Library
6.1.3 (situation should be the same up to 6.2.6, tough)
Version of OS(s) listed above with issue
Android 6.0, 7.0, 9.0 with HW: Sony Experia, ZTE Blade A512, couple of Samsung devices, Nokia (model not known at the moment)
Hi there,
thanks for your work and the library so far! : - )
This is neither an actual bug report nor a real feature request. I wanted to ask about and compare throughput values achieved by the current state of the BluetoothLE library. I wanted to find ideal configurations for the connection.
In short: What are real-world throughput values for write operations? Due to the way the bluetooth stacks operate the measured values ideally transfer a larger amount of data to another device. For example: 100kB of data split into 20b packets. The measured times for a single write operation might be interesting as well despite being somewhat difficult to interpret.
I expected values around 2.5kB / s which is already very conservative compared to various iOS guides, Android forums and Nordic chip forums.
Using defaults for PauseBetweenInvocations (100ms) and write WITH response: ~0.125kB / s
Setting PauseBetweenInvocations to 0ms and write WITH response: ~0.4kB / s
Using defaults for PauseBetweenInvocations (100ms) and write WITHOUT response: 0.2kB / s
Setting PauseBetweenInvocations to 10ms and write WITHOUT response: ~1.8kB / s but with the high risk of overflowing bluetooth stack buffers internally and risking crashes on different platforms.
Setting PauseBetweenInvocations to 0ms and write WITHOUT response: almost guaranteed exceptions during writing to the characteristic
Note that I'm in control of the FW of the other bluetooth device so I can check parameters such as the interval (which is very small at only 9 * 1.25ms = 11.25ms). It seems that there is no real room for optimization of the parameters from the Android side so far. Setting the ConnectionPriority for example does not "help" on my test devices since the interval is already very short (on some devices even at 7.5ms which is the shortest possible according to the specs). Even when we account for only 4 MTUs within one interval we would get a theoretical value of 4 * 80b / 7.5ms => ~11kB / s
I would like to see a setup using BluetoothLE that achieves higher throughput than I did. Maybe there is still something wrong here?
Kind regards,
Mike
Steps to Reproduce
Since it is about throughput and performance: Every example that measures the data transfer rates should be suitable
Expected Behavior
Throughput of roughly 2.5kB / s which should be very conservative compared to various calculations found in Nordic forums, for example.
Actual Behavior
The throughput is way below the expected values. The details are given in the issue description above.

Great tests and I appreciate the numbers.  I'm afraid I don't have anything in terms of numbers to compare against, but I can give you a few of hints.

You should always do write with response.  I know it slows things down, but writes do tend to fail.
If you use one of the WriteBlob functions, it will use transactions and tight loop.  It can help a bit
On Android, you should run RequestMtu to a max rate of 512 (you will never get this value, but it should be higher than the default 20 bytes).  iOS will auto-negotiate this value.

That all being said, if you need any sort of performance - BLE is not your guy.  You are better off going classic Bluetooth.
I usually don't use BLE if I need any sort of performance.  I tend

Thanks for the response. Yes BLE is definitely not designed for high throughput. Nevertheless, tasks such as updating the firmware of a low energy device need to achieve data rates that are sufficient for the user. Talking about 0.1 to 0.2kB / s is simply not enough even for small images of 150kB.
Choosing an MTU that is larger 20 byte is not recommended right now since there are tons of devices out there that do not allow this feature yet. This, of course, will change in a few years from now. In general you are completely right that the MTU should be negotiated between the devices but for now I focus my experiments on the generation of phones between 2015 and 2017.
My following thoughts are closely related to PauseBetweenInvocations, what does it means? #252
Writing larger data WITH response slows down the process so much that it's almost unusable even for smaller firmware updates. Using writes WITHOUT response should work on Android as well as on iOS from what I've seen so far.
Android: The usual callback (that is called when a response is received after a write) should also be called when writing WITHOUT responses. It usually simply means that the buffer inside the stack has enough room again for some data. Using this callback mechanism should optimally utilize the data rate as discussed in this stackoverflow post. I think that there might be Android versions in combination with HW devices that still require the write WITHOUT followed by write WITH response hack as in iOS (see below).
iOS: On iPhones it seems that the callback is not called correctly though. There should be 2 approaches to fix this. The one is a check for a flag that tells if data can be transmitted again. Due to a lack of documentation this might not always work or might be tricky to implement in a stable fashion. So the second approach is to write WITHOUT responses for a few times followed by a write WITH response as discussed above for Android.
What I have right now with the current state of the library:
Note that the following measurements were done on a ZTE A512 with Android 6.0 which chooses 9 * 1.25 = 11.25ms for the interval during my transmission tests which is quite a good value compared to my other test devices (Sony Xperia chooses 40 * 1.25 = 50ms which sucks!).
I use 10ms for PauseBetweenInvocations. Whenever I send data packages I do it by using writes WITH response to be safe in the beginning. In case that larger data such as firmware images (more than 1 MTU) has to be sent, all consecutive writes of the same data blob are written as 3 writes WITHOUT and every fourth WITH response. This way I achieved a stable transmission with roughly 0.5kB / s. Note that 1 vs. 3 was chosen since 4 packets per interval are very common for phones. Empirically speaking this feels quite safe to do but one could increase that to probably 1 vs. 5 (got 0.64 kB / s this way) or something like that. Also note that these ratios do not actually relate to the number of packets according to any documentation! It's just about experiments and best guesses for anyone playing around with it!
Now my suggestions for the BluetoothLE library:

Android and iOS: Remove the utilization of the PauseBetweenInvocations for all writes WITH response since they should not be needed at all. This is actually a problem since it slows down the writes unnecessarily.
Android: Implement the recommended callback mechanism for writes WITHOUT response similar to writes WITH response but keep the configurable delay in there for everyone that needs it. Every API user can then implement the swapping between writes WITH and WITHOUT responses as they wish.
iOS: Maybe implement the checking of the flag but this might not be needed when implementing the first two suggestions.

iOS doesn't use the queue or pausing mechanism at all, so it won't be a factor here.
You are correct, I can remove the PauseBetweenInvocations for Write WITH Response, but this requires a bit of work.  The callback for WriteWithoutResponse has not proven to work with a bunch of devices I have.  I'll play around with it again.
Thanks for the great examples and digging.  This is helpful.  I'll see what can be done

Thanks, sounds great! I'll be happy to test any changes you make on my available devices and compare throughput and stability with what I have so far to post results here again : - )

I have been quite active on trying to work this out.

I've restored the iOS functionality for Write without response.  It does get up to 2.5 kbs
Android has had its Pause functionality removed without any issues so far
Android has a working write without response callback now

iOS did get some positive benefits.  Android on the other hand is still subpar in speed (at least on my device).  I did get a much better result, but still only 1.3 kbs on average which is 1/2 of iOS.
You can find these efforts here:
https://github.com/shinyorg/shiny/tree/dev/src/Shiny.BluetoothLE and a decent testing sample here:
https://github.com/shinyorg/shinysamples/tree/master/Samples/BluetoothLE
I'd appreciate if you could run your tests with the new library.  I will backport this functionality to this library if needed and I can get it to decent speeds.

Hey, thanks for the effort! Right now I'm only using BluetoothLE without any Shiny integration so I would need to invest some time there. I will report back here as soon as I have new test results.

You actually don't have to put any investment in.  The samples I linked contain almost everything you need to test with.  Since you have a remote device and different types of android devices, I want to see if things have improved (it also has toggles to flip things off - queuing, main thread, etc).

I wanted to compare more or less the same scenarios after your changes so I decided to integrate Shiny to replace BluetoothLE temporarily. I know that the direct comparision cannot be made anymore due to internal changes but still my measurements are very rough in the first place anyways. I wanted the surroundings to stay the same.
Stability in Android (Sony Android 7, Samsung S7 Android 8, ZTE A512 Android 6) :

Using writes WITH response worked as stable as before.
Using writes WITHOUT response worked stable across all my data transfer experiments. Congrats so far!

Performance in Android (same devices as above):

Using writes WITH response has similar performance to 6.2.6. of BluetoothLE (but see comment below).
Using writes WITHOUT response has a HUGE performance impact! A 230KB firmware update took roughly between 3:00 and 4:30 with even larger connection intervals chosen by the OSes. From what I saw so far I would say that the data rate is 2 to 4 times faster than before! Congrats again!

iOS: Sorry, I did not perform any experiments here.
A note that I wanted to post here a week ago: I upgraded to BluetoothLE 6.2.6 from 6.1.3 and I noticed a performance hit of roughly 15-20%. Unfortunately, I didn't find the time yet to check the GIT history to maybe nail the impact down to a specific change. It might not be that important though if you could backport your fixes from Shiny to BluetoothLE. I would then perform in-depth experiments with all of my devices on Android again. I could also include 3 different "new" iOS devices that have become available to me.

Hi @Maze112 - Sorry for the delayed response.  I wanted to thank you for the excellent testing and great feedback.  It is very rare that I get these kind of posts and I really do appreciate them.  As such, I have made the task of porting to shiny almost minimal at this point if you try out the latest version.  I've documented how to skip most of the dependency injection portion here: https://shinydocs.azurewebsites.net/docs/nodi .  If you still find this problematic, let me know and I'll be happy to a) do more to assist with the porting in terms of how the library is used and/or b) back port to this library.
Also, I haven't given up trying to further improve perf on Android.  The opportunities on how to do this at this point are difficult at best.
I'm going to close this issue, but feel free to respond when you have a moment

Thanks so far!
I understand that you would like your library users to make the transition to Shiny but the "beta" state is holding me back a bit ; - ) Not sure how I will proceed yet.
Having stable 1.3KB/s in Android should be fine : - )

That all being said, if you need any sort of performance - BLE is not your guy. You are better off going classic Bluetooth.

I've been working to integrate this package into my project for a while now and was very optimistic about it, but only recently got to the point where I was able to reliably measure performance. I was really confused by what I was seeing in terms of performance and then found this thread. I realize now that I had been making the assumption that this package was compatible with all modern bluetooth versions/flavors up to and including BLE, I did not realize that it may be exclusively for BLE. This thread makes it seem like my understanding of the different versions and flavors of bluetooth that exist may have been incorrect. Is it true that this package only works for the low energy flavor of bluetooth and as such is limited to these low performance rates and CANNOT reach the bluetooth "classic" performance rates?

BLE cannot come even close to reaching classic rates.  It wasn't designed too.  And yes, this package only supports BluetoothLE, hence. the name


Cache Layer according to "Pipfile.lock" or "poetry.lock".

Considering that layer contains only the dependencies, we could just cache e avoid redeploy according to "Pipfile.lock", it is possible?

I'm not sure if my case is the same or needs a new ticket.
We're using dockerizePip: true. Running serverless package multiple times locally will result in about 19s packaging time. The CI process takes about 5min in total incl deployment, but a large chunk is packaging. I assume this is due to the docker build process, as "docker" is not being executed locally in consequent runs.
Log from CI:
...
Generating requirements.txt from "pyproject.toml"
Parsed requirements.txt from pyproject.toml in /home/runner/work/helios/helios/.serverless/requirements.txt
Installing requirements from "/home/runner/.cache/serverless-python-requirements/df293b6184a94fbfd982a4f95fea1698fe858cb805f178dd2112d29905e6a12e_x86_64_slspyc/requirements.txt"
Docker Image: sls-py-reqs-custom
Using download cache directory /home/runner/.cache/serverless-python-requirements/downloadCacheslspyc
Running docker run --rm -v /home/runner/.cache/serverless-python-requirements/df293b6184a94fbfd982a4f95fea[16]...
Python executable not found for "runtime": python3.9
...

Config:
  pythonRequirements:
    slim: true
    # If changes do not reflect, use `serverless requirements cleanCache`
    # See See https://github.com/serverless/serverless-python-requirements/issues/437
    slimPatterns:
      - "**/tests/**"

      # Removal of boto3 due to Lambda size limitation.
      # This should ideally be packaged with the app.
      - "**/botocore/**"
      - "**/boto3/**"

      # Django files we don't need for the API.
      - "**/locale/!(en|en_*)/**"
    strip: false
    dockerizePip: true
    dockerFile: ./deploy/Dockerfile-requirements
    #dockerImage: public.ecr.aws/sam/build-python3.9
    zip: false
    # Set to `true` to cleanup cache in case of requirements issues.
    invalidateCaches: false
    useDownloadCache: true
    useStaticCache: true
    layer: true

Is there a current solution to speed up or circumvent the docker build if requirements have not changed?
Are there ideas about a future solution to reduce the runtime if requirements have not changed?


Getting error while exporting file as XML

Getting error while exporting file as XML
After doing the word, while I was trying to export the file as XML I' getting the following error:
Python: Traceback (most recent call last):
  File "C:\Users\HP\AppData\Roaming\Blender Foundation\Blender\3.4\scripts\addons\mitsuba-blender\io\__init__.py", line 136, in execute
    self.converter.scene_to_dict(deps_graph, window_manager)
  File "C:\Users\HP\AppData\Roaming\Blender Foundation\Blender\3.4\scripts\addons\mitsuba-blender\io\exporter\__init__.py", line 97, in scene_to_dict
    geometry.export_object(object_instance, self.export_ctx, evaluated_obj.name in particles)
  File "C:\Users\HP\AppData\Roaming\Blender Foundation\Blender\3.4\scripts\addons\mitsuba-blender\io\exporter\geometry.py", line 131, in export_object
    mts_mesh.write_ply(filepath)
AttributeError: 'NoneType' object has no attribute 'write_ply'

To Reproduce
Steps to reproduce the behavior:

Go to 'File'
Click on 'Export'
Scroll down to 'Mitsuba(.xml)'
See error

Environment (please complete the following information):

OS: Windows 11
Blender Version: 3.4
Add-on Version: Latest

Additional context
I was trying to do my work on Blender 4.0. But I faced difficulties while adding the addon. After reading the issues, I downgraded to version 3.4 and successfully add the addon. But after completing my word I got the error while exporting.

This should now be fixed with the latest release of the add-on. Please upgrade and re-open this issue if the problem persists.