Refactor action codec

This change will be needed to accommodate alarms implementation.
Alarms are emitted from the data-plane (follower) to the leader. The leader manages the state of alarms. A channel to communicate the leader and the follower will be needed, similarly to the config actions mechanism currently implemented.
Thus I'm splitting action_codec.lua into codec.lua and action_code.lua.
codec.lua will hand only the encoding/decoding operations of different data types.
action_codec.lua defines the verbs and arguments of the action messages.
In addition to the refactoring, the PR fixes an issue in codec.lua. It's possible to encode a table using config, however the config method expects a config argument which in the case of a table would be nil or an empty table. I think that makes the interface more cumbersome, so I'm defining a 'encode:table' and decode:table primitive.

It's more efficient to encode compound data structures using the yang.compile_data_for_schema_by_name.  Could you use this?
I am a little skeptical about the attempt to reuse code here; the codec is a small file that serves a special purpose, and trying to go generic doesn't sound like the right thing to me.  In particular I think the data plane should not be creating new files using the encoder:config approach; instead, its data should be small enough to fit directly in the encoded message.

@wingo If I understood you correctly, you prefer that both alarm_codec.lua and action_codec.lua implement their own encoder and decoder although the code for the data type is going to be the same?
Another doubt I have is regarding compound data structures. yang.compile_data_for_schema still compiles data to a file AFAIU. That was the only way I found to encode tables. The point is that alarms API have tons of parameters and it's more convenient to pass a table or tables as argument than scalar data types. Is there any way to encode a table without compiling to a file? That's what I would ideally need.

Taking questions one by one.
Firstly regarding code sharing / etc.  Well.. I don't know, I just wonder if configuration actions and alarm notifications are going to end up sharing that much.  Config changes have to be quite large so they by default write some info out to disk, and they have that legacy facility for encoding ad-hoc tables.  Alarm notifications will be small, have no legacy burden, and they come from the dataplane during normal operation so they shouldn't touch disk.
Concretely to make this work you need to add the ability to serialize an inline instance of YANG data without going to disk, and you don't need the ability to serialize a class, to serialize YANG to a file, or serialize an ad-hoc table to a file.  I think you will end up sharing 50 lines of code but also having to define and support its public interface; could be OK but for me I would punt.
Secondly regarding how to compile data to a byte stream: use the data_emitter interface in binary.lua, or rework data_compiler_from_grammar; but can you point me to examples of what notifications you would emit?  How much information does a notification contain?  If there are only 3 (say) different ways you can notify a change to an alarm, then we should special-case them I think.

Regarding code sharing, I agree with all the points. A big chunk of shared code had to do with class serialization which if it should be avoided in alarms so operations run fast and do not touch, makes the refactoring less valuable. All I need for data encoding in alarms are strings, ints and tables.
About the type of notifications that alarms emit, so far what I have in the alarms WIP PR (https://github.com/Igalia/snabb/pull/871) are raise and clear, which likely will be all the messages the follower needs to pass to the leader. All the other alarms operations are RPC calls addressed directly to the leader (set-operator-state, purge-alarms and compress-alarms).
Besides raising and clearing an alarm can change its status. An alarm changes its status if there's a change in severity, clearance status, and alarm-text update. There won't be a status-change message but simply a raise operation with arguments. The arguments may yield a status change so the alarm will be updated.
Basically an alarm attributes consist of two groupings of attributes [1]:

common-alarm-parameters (resource, alarm-type-id, alarm-type-qualifier, alt-resource, related-alarm, impacted-resource, root-cause-resource).
resource-alarm-parameters (is-cleared, last-changed, perceived-severity, alarm-text, status-change).

IMHO the most relevant are the ones that imply a status-change: is-cleared, perceived-severity and alarm-text. Then resource, alarm-type-id and alarm-type-qualifier define an alarm key, so they are needed too. Other attributes are managed by the domain logic: status-change and last-changed. All the other ones could be defined statically: alt-resource, related-alarm, impacted-resource and root-cause-resource.
All in all, my definition of raise alarm is https://github.com/Igalia/snabb/pull/871/files#diff-6796341aa7d23a74325a26341d76765bR16
function alarms.raise_alarm (codec, key, args)
   key = codec:table(key)
   args = codec:table(args)
   return codec:finish(key, args)
end

Where key is a table formed by {resource, alarm_type_id and alarm_type_qualifier} and args is a table which may contain at least perceived_severity and alarm_text. The clearance status will be set by raise (is_cleared=false) or clear (is_cleared=true) operations.
[1] https://tools.ietf.org/html/draft-vallin-netmod-alarm-module-02

Superseded by #877


Gitsigns don't see vim.env change anymore

Description
I'm using vim.env.GIT_DIR and vim.env.GIT_DIR to tell Neovim where is my git bare repo for dotfiles.
More about this and the code snippet here.
Since commit 19654d9 it stopped working with Gitsigns, only works with Fugitive.
From this commit and later I can't see any git signs, can't navigate via hunks, etc. when I'm using bare git repo.
I printed vim.env.GIT_DIR from command mode and saw that the value is correct.
The problem is related to a specific commit.
Neovim version
v0.9.2
Operating system and version
Arch Linux
Expected behavior
git repo detected by vim.env.GIT_DIR and vim.env.GIT_DIR
Actual behavior
git repo not detected
Minimal config
It's not easy to do, but I will make it later if required...

Steps to reproduce
-- set these variables, cd to home and open some changed git-tracked file
vim.env.GIT_DIR = git_dir
vim.env.GIT_WORK_TREE = home

Gitsigns debug messages
No response

@coffebar can you try #896

works great, thanks!

Just to note this fix won't be required in Nvim 0.10.


How to get Torch 2.5.0.dev and TensorRT 2.5.0.ev

I'm trying to updated to the latest version using:
F:\Hybrid\64bit\Vapoursynth>python -m pip install --pre torch_tensorrt --index-url https://download.pytorch.org/whl/nightly/cu124
Looking in indexes: https://download.pytorch.org/whl/nightly/cu124
Requirement already satisfied: torch_tensorrt in f:\hybrid\64bit\vapoursynth\lib\site-packages (2.4.0.dev20240518+cu121)
Requirement already satisfied: torch<2.5.0,>=2.4.0.dev in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch_tensorrt) (2.4.0.dev20240523+cu121)
Requirement already satisfied: tensorrt<10.1.0,>=10.0.1 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch_tensorrt) (10.0.1)
Requirement already satisfied: packaging>=23 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch_tensorrt) (24.0)
Requirement already satisfied: numpy in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch_tensorrt) (1.26.4)
Requirement already satisfied: typing-extensions>=4.7.0 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch_tensorrt) (4.12.0)
Requirement already satisfied: tensorrt-cu12 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from tensorrt<10.1.0,>=10.0.1->torch_tensorrt) (10.0.1)
Requirement already satisfied: filelock in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (3.14.0)
Requirement already satisfied: sympy in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (1.12)
Requirement already satisfied: networkx in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (3.3)
Requirement already satisfied: jinja2 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (3.1.4)
Requirement already satisfied: fsspec in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (2024.5.0)
Requirement already satisfied: mkl<=2021.4.0,>=2021.1.1 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (2021.4.0)
Requirement already satisfied: intel-openmp==2021.* in f:\hybrid\64bit\vapoursynth\lib\site-packages (from mkl<=2021.4.0,>=2021.1.1->torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (2021.4.0)
Requirement already satisfied: tbb==2021.* in f:\hybrid\64bit\vapoursynth\lib\site-packages (from mkl<=2021.4.0,>=2021.1.1->torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (2021.12.0)
Requirement already satisfied: MarkupSafe>=2.0 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from jinja2->torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (2.1.5)
Requirement already satisfied: mpmath>=0.19 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from sympy->torch<2.5.0,>=2.4.0.dev->torch_tensorrt) (1.3.0)

F:\Hybrid\64bit\Vapoursynth>python -m pip install tensorrt==10.0.1 tensorrt-cu12_bindings==10.0.1 tensorrt-cu12_libs==10.0.1 --extra-index-url https://pypi.nvidia.com
Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
Requirement already satisfied: tensorrt==10.0.1 in f:\hybrid\64bit\vapoursynth\lib\site-packages (10.0.1)
Requirement already satisfied: tensorrt-cu12_bindings==10.0.1 in f:\hybrid\64bit\vapoursynth\lib\site-packages (10.0.1)
Requirement already satisfied: tensorrt-cu12_libs==10.0.1 in f:\hybrid\64bit\vapoursynth\lib\site-packages (10.0.1)
Requirement already satisfied: tensorrt-cu12 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from tensorrt==10.0.1) (10.0.1)
Requirement already satisfied: nvidia-cuda-runtime-cu12 in f:\hybrid\64bit\vapoursynth\lib\site-packages (from tensorrt-cu12_libs==10.0.1) (12.5.39)

F:\Hybrid\64bit\Vapoursynth>python -m pip install -U vsrife==5.2.0
Collecting vsrife==5.2.0
  Using cached vsrife-5.2.0-py3-none-any.whl.metadata (1.8 kB)
Requirement already satisfied: numpy in f:\hybrid\64bit\vapoursynth\lib\site-packages (from vsrife==5.2.0) (1.26.4)
Requirement already satisfied: requests in f:\hybrid\64bit\vapoursynth\lib\site-packages (from vsrife==5.2.0) (2.32.2)
INFO: pip is looking at multiple versions of vsrife to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement torch>=2.5.0.dev (from vsrife) (from versions: 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1)
ERROR: No matching distribution found for torch>=2.5.0.dev

using the calls from the README.md, but I only get Torch 2.4.0.dev and TensorRT 2.4.0.dev and thus can't install vsrife 5.2.0.
Any advice how to get this working?

When you already have a package installed and you want to upgrade it, you have to manually append -U or --upgrade in pip.

Thanks! Hopefully this does not break other stuff. :)


comment functionality

Functionality for the comment section. Features include:

"Pagination".. the pagination is not actually loading in chunks from the server, but just loads the whole json array at once and displays 12 at a time until it's done. Will work on actual pagination. This is a 3/10 implementation just to do for now.

Invalid input checking

Only ASCII values allowed in name
The words "seun" or "computabeast" cannot be included in the name input #censorship
Name has 25 chars max and must be non-empty
Body input has 150 chars max and must be non empty

Video:
https://drive.google.com/file/d/12ELuu3bOFwixY2hqV3k-8TXezdYpM_lX/view?usp=sharing

HTML injection is also taken care of with my method


Use eslint-plugin-simple-import-sort instead of import/order

https://github.com/lydell/eslint-plugin-simple-import-sort

resolved by #144


I'm going to have to downgrade from 1.14 - Hangs every 10 - 15 minutes.

Upgraded to 1.14 when it available today. Since then I get about 10 - 15 to work and then I get the message (attached).
Going to figure out how to downgrade to the previous version.

VSCode Version: 1.14
OS Version: Windows 10 Pro 1703 / Build 15063.413

Steps to Reproduce:

Upgrade to 1.14
Use the editor for a few minutes

Confirmed that is also breaks the Mac build of 1.14 this one really screwed the pooch.

Does this reproduce without extensions? Launch with code --disable-extensions to check.

@mdenson-dayspring Also: What kind of files are you working on? Could you attach a screenshot of the window when it is frozen?

I'm getting this same error on Ubuntu 16.10. I can interact with code when I run code --disable-extensions; however, if I disable all installed extensions and restart code, then code will still hang like the OP's issue.

@tuxagon When starting with code --disable-extensions extensions can only be uninstalled, but not disabled. Do you by chance have atefth.ruby-on-rails-snippets-0.0.1 installed and does uninstalling it resolve the issue?

I experienced the same issue when working on a NativeScript project.  Editing TypeScript files would cause VSCode to hang after just a few key presses.  Rolled back to 1.13.

Likely #30449, where the TypeScript service eats the CPU. Try https://github.com/Microsoft/vscode/issues/30449#issuecomment-314666925 to help investigate / as a workaround.


Can't load saved GraphSage Model

Describe the bug
I trained a GraphSage model and saved it using model.save('model.h5'), but when I try to load it using tf.keras.models.load_model('model.m5') I get an error.
To Reproduce
Steps to reproduce the behavior:

Create a GraphSage Model
Save its weights'
Load them
See Error

Observed behavior
ValueError: Unknown loss function: sigmoid_cross_entropy_with_logits_v2
Expected behavior
Load model correctly.
Environment
Operating system: Google Colab

Hi @rmant you can use model.save_weights instead of model.save and then use the model.load_weights to load the weights.


How to add column to AspNetRoleClaims I cant register  IdentityRoleClaim via constructor to add more fields

My main issue: I want to add a property to IdentityRoleClaim called description.
Why? So that when I build the front end I can display friendly text as to what the claim is all about.
My current work around: I basically extended the role and have to map it to my CustomIdentityRoleClaim. My dba's are complaining....
 public  ICollection<CustomIdentityRoleClaim<TKey>> Claims { get;set; } ;

Before asp.net identity 2 I was able to pass a little bit more information that would have helped me customized my tables.
public class IdentityDbContext<TUser, TRole, TKey, TUserLogin, TUserRole, TUserClaim> : DbContext
        where TUser : IdentityUser<TKey, TUserLogin, TUserRole, TUserClaim>
        where TRole : IdentityRole<TKey, TUserRole>
        where TUserLogin : IdentityUserLogin<TKey>
        where TUserRole : IdentityUserRole<TKey>
        where TUserClaim : IdentityUserClaim<TKey>
    {...

Today
  public class IdentityDbContext : IdentityDbContext<IdentityUser, IdentityRole, string> { }

Which allows me to declare my own types of IdentityUser + IdentityRole very nice
    public class ApplicationDbContext : IdentityDbContext<ApplicationUser, ApplicationRole,string>
//Which basically inherit from it and add more properties
public class ApplicationUser : IdentityUser<string>
    {
        public int CustomProperty { get; set; }
        public ApplicationUser() {

        }

    }
    public class ApplicationRole : IdentityRole<string>
    {

        public ApplicationRole()
        {

            this.Id = Guid.NewGuid().ToString();
        }

        public ApplicationRole(string name)
            : this()
        {
            this.Name = name;
        }

        public string Description { get; set; }
        // Add any custom Role properties/code here
    }

But what happened to public class IdentityRoleClaim where TKey : IEquatable namely why can't I inject my own NOTICE the IdentityRoleClaim
I would suggest the following
  public class IdentityDbContext : IdentityDbContext<IdentityUser, IdentityRole, string,IdentityRoleClaim > { }

Besides downloading your source and adding a property to IdentityRoleClaim I don't see a way to add to a new property to the field.

Same problem here ... Where is the option:
public class IdentityDbContext : IdentityDbContext<IdentityUser, IdentityRole, string, IdentityUserLogin, IdentityUserRole, IdentityUserClaim>

Please add this pull request !https://github.com/aspnet/Identity/pull/643

Duplicate of #585


Few-shot learning

Hello, good day. I am trying to build an application using the library you've developed.
Each method should have its own prompt. The prompt from the user should run my method prompt first. Then the method should work and I should get an answer.
I have read the README file, but I am having trouble achieving this. Could you please assist me?

could you share your demo repo on GitHub?  and I can take a look. Thanks

For example, when the send_email method runs, I want to run a prompt where I want it translated to German. When the read_email method is run, I want it to be translated into Turkish and run this prompt before the method.
https://github.com/YahyaAltun/chatbotdemo

@YahyaAltun  how about Spring AOP?  you can use AOP to intercept any method call.

I tried spring aop but it didn't give the result i wanted. Chatgpt answered the question when this line;
// display reply combined text with function call
System.out.println(response.getReplyCombinedText());
I want to manipulating chatgpt answers. How can i use prompt.properties for to add a prompt to the user prompt. Thanks

I got it and can i ask another question?
The conversation consists of only one question and one answer. I want it to respond to my new question by recalling the previous answer and the previous question.(as in chatgpt)
Should I save the questions and answers to the database and send them in every new request, or is there another way to do it?


Update Azure deployment example

What I am changing

I've updated the Azure deployment documentation and sample to reflect changes to the titiler codebase (https://github.com/developmentseed/titiler/pull/537), fixes to upstream dependencies (starlette), and updates to the Azure Python function wrapper package.
How I did it

The updates to the Azure sample match the main sample application. The others were made through trial and error.
How you can test it

No tests for these examples, run as-is.
Related Issues

changes to the sample required by changes made here: https://github.com/developmentseed/titiler/pull/537
re-enabling of middleware based on this issue being resolved: https://github.com/encode/starlette/issues/1320

thanks @gassc, as I have no way to test this I will have to trust you 🙏 😄


Remove dataset add_parameter

With the merge of #1477, the method add_parameter of the DataSet became obsolete. This PR removes the method from the DataSet API by breaking it. Removing that method will in turn make development of the interdependencies stuff easier, since the method extend_with_paramspec will no longer have to exist.
If you wish, you can view this PR as the first "clean up the fallout of #1477" PR. There are more to come, including schema upgrades etc.
~Still pending: upgrade all tests to use set_interdependencies instead of add_parameter.~
I am open to input on how to best break/remove/deprecate DataSet.add_parameter, but I stress the point that its existence is a bit of a roadblock and that I therefore find it justified to simply kill it without going through a deprecation cycle. Looking ahead, we will need to break a lot of stuff and I think we should allow ourselves to move quickly.
@QCoDeS/core this is now ready for review.

@QCoDeS/core, bump, bump. All the commits here are contained in #1555, but it might be good to get this in first. We can also do both in one go. What do you think?

Let's do these PRs one-by-one. I'll have a look at this tomorrow.

I am not super keen on removing code that is public api without a deprecation cycle but I don't have a good solution. Should we check that this is not called from any code that we know of before doing this?

Code that i know (and i checked) of does not use DataSet.add_parameter.

No occurrences in main qcodes (obviously) nor in the current master of qdev-wrappers.


property getBoundingClientRect. Property not found in Node

Flow flags an error when calling getBoundingClientRect on an element:
const rootElement = document.getElementById('root')
const { width, height } = rootElement.parentNode.getBoundingClientRect()

src/index.js:19
 19: const { width, height } = rootElement.parentNode.getBoundingClientRect()
                                                      ^^^^^^^^^^^^^^^^^^^^^ property `getBoundingClientRect`. Property not found in
 19: const { width, height } = rootElement.parentNode.getBoundingClientRect()
                               ^^^^^^^^^^^^^^^^^^^^^^ Node

My guess is that flow is suggesting that parentNode could be undefined, but in this case it would fall back to document.body.

Try parentElement--not all Nodes are Elements, and getBoundingClientRect is an Element method.

thanks @rjbailey, I think that should do it.
@meleyal: please reopen if that doesn't work

Thanks @rjbailey, that worked :+1:


Investigate Destination service sending address updates without labels

If the Destination service receives an address update, and fails to find an associated pod, it still sends an update message to the Proxy:
https://github.com/runconduit/conduit/blob/e1e1b6b599f99ce05221c4680e9ed890eda3fa2e/controller/destination/listener.go#L81-L86
We should determine if this is advisable behavior, and also confirm whether this can ever actually happen. A first step could be to add a metric counter that increments any time this happens, in addition to logging an error.
source: https://github.com/runconduit/conduit/pull/781#discussion_r182215953

Fwiw, @olix0r ran into this issue this morning, and I think it's a result of the Kubernetes API cache not being populated before serving requests from the proxies. We should fix this issue by completing the fix described in #1013.

I think this was fixed in #1168 but I don't have a great way to verify it unfortunately. I'll leave it open for now, to see if we get any more reports of it.


Original hostname: 'redis' failed to resolve 'redis.svc.cluster.local' In k8s

2022-02-06T11:21:15.662Z,[redisson-netty-2-23],ERROR,org.redisson.connection.DNSMonitor,112,Unable to resolve redis, io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Search domain query failed. Original hostname: 'redis' failed to resolve 'redis.svc.cluster.local'
at ...
Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/<IP_ADDRESS>:53] query via UDP timed out after 5000 milliseconds (no stack trace available)

redisson   3.16.0
netty        4.1.43
redis server:  redis 5.0.6

did you try to update netty and/or check your dns server?

Define fully qualified domain.


move unstable nightly aeron builds into their own workflow

So, that we can get the main nightlies green as the Aeron tests are the ones failing the most.

@jrudolph You fine with this being merged?

lgtm, but quick question since I am on mobile, why weren't the nightly tests running in the first place (did we explicitly disable them or?)

All tests have been running but no one is closely monitoring their result afaik. (We should enable mails for failed runs at some point, the question is where those should go. Maybe a new mailing list just for that?)

All tests have been running but no one is closely monitoring their result afaik. (We should enable mails for failed runs at some point, the question is where those should go. Maybe a new mailing list just for that?)

I am guilty of this. Mailing list is a good idea, another one is to add a badge onto README.md specifically for nightly tests

I don't think anyone is guilty here :) It's super difficult to deal with the pekko core tests because they are so hard to stabilize because of the non-determinism involved. (Other projects remove sources of non-determinism to avoid this problem, however, that means that tests are less representative of real world scenarios unless you manage to create deterministic versions of all the race conditions that could happen in reality which might work for unit tests but the bigger the scenarios get the harder it gets...)
So, even in the best case, there will be quite a bit of spam involved. The difficulty is to find a good signal to noise ratio (which has been a much discussed topic in the past because of the amount of work required to get there...).
The thing we are interested in is finding regressions. In the best case, a regression will make some test fail deterministically after a change but some tests could also start to fail non-deterministically in a regression case. These days, in many cases, the tests themselves are accidentally depending on some outcome that is not guaranteed.
In the best case, all the flakiness could be fixed by hardening the tests (and this an ongoing effort) but this is almost a job in itself. The next best solution is to identify tests that started failing at some point. In the past, I wrote different kinds of tools to deal with that (collect test results over a period and then trying to extract the signal from the noise) which was somewhat easy because Jenkins would parse JUnit XML results and provide them as json.
Another strategy is to separate flaky and stable tests and run them in different jobs (which already happens by tagging tests) to make sure the stable tests always succeed (but then the question is who would watch the unstable tests?).
In any case, it's not a simple problem.

Regarding the visibility point, I just created a PR which adds a badge on README.md showing current status of aeron nightly tests

I don't think anyone is guilty here :) It's super difficult to deal with the pekko core tests because they are so hard to stabilize because of the non-determinism involved. (Other projects remove sources of non-determinism to avoid this problem, however, that means that tests are less representative of real world scenarios unless you manage to create deterministic versions of all the race conditions that could happen in reality which might work for unit tests but the bigger the scenarios get the harder it gets...)

Indeed, I just manually ran the tests and noticed that one of them failed because they the test didn't get the necessary amount of messages (test received 160 and expected at least 200). I would suspect that one reason behind the flakiness is the strength of the machines, another Apache project that I work on (Kafka) has the same problem where due to the machines being so weak it causes tests to fail.
One way to solve this would be to use our own self hosted runners, which would make sense for nightly tests. @pjfanning did some research into this and Apache supports this albeit there are some security workarounds we have to do (iirc some of those workarounds are irrelevant for nightly tests, they are only a concern on PR's).

I would suspect that one reason behind the flakiness is the strength of the CI machines, another Apache project that I work on (Kafka) has the same problem where due to the machines being so weak it causes tests to fail.

This is part of the reason. We always thought of that as a feature, though. By putting more stress on the tests, we hoped to make them more resilient. Until GHA was adopted all the tests used to run on ~10 year old servers. The multi-node tests also ran on these machines and after the move to GHA needed dedicated VMs to run on (because the tests need ssh access to set them up).
One problem it might not be slow build machines which makes the tests fail in the first place but a high variance in processing power (e.g. for GHA because they run on unknown shared infrastructure with an unknown amount of noisy neighbors).

This is part of the reason. We always thought of that as a feature, though. By putting more stress on the tests, we hoped to make them more resilient. Until GHA was adopted all the tests used to run on ~10 year old servers. The multi-node tests also ran on these machines and after the move to GHA needed dedicated VMs to run on (because the tests need ssh access to set them up).

One problem it might not be slow build machines which makes the tests fail in the first place but a high variance in processing power (e.g. for GHA because they run on unknown shared infrastructure with an unknown amount of noisy neighbors).

Yes, both of these go hand in hand. Can't speak to much of Akka/Pekko nightly tests because I am not that familiar with it, but for Kafka because the way the tests are run is also not deterministic (irrespective of machine strength), you can hit cases where a test happens to be running alongside a lot of other tests. and because of that the specific test has a lot less resources which causes it to fail. One very common workaround to solve this issue is to increase the timeout which does solve this specific problem but then creates another problem which is when people run the tests locally they complain that they take too long because there is a 5 minute timeout (which was set to 5 minutes because of flaky CI issues) albeit this can be solved by just configuring the timeout separately in CI vs locally.
On the point of taking advantage of using slow hardware deliberately to test the resiliency/memory usage of Pekko while I can understand the sentiment this to me sounds like we are conflating too many things in a test which then creates the scenario where by definition they will also be flaky (especially if, as you said, we don't even know the strength of the machines that the CI is run on).
Would it be wiser to separate the tests out by their primary concern so that we can better handle them? i.e. using self hosted runners for general nightly tests makes sense since we need the raw power and if we also want tests with deliberately less powerful hardware for resilience to me it seems like selecting the relevant subset of tests and running them on a dedicated machine inside LXC/docker so we can more precisely limit the resources of the test to help with determinism?

you can hit cases where a test happens to be running alongside a lot of other tests that happen to be expensive and because of that the specific test has a lot less resources which causes it to fail

Running tests in parallel is turned off for that reason.

One very common workaround to solve this issue is to increase the timeout which does solve this specific problem but then creates another problem which is when people run the tests locally they complain that they take too long because there is a 5 minute timeout (which was set to 5 minutes because of flaky CI issues) albeit this can be solved by just configuring the timeout separately in CI vs locally.

This is also already done by setting timefactor=2 when running tests in CI.

they complain that they take too long

Comprehensive tests are already 1 hour+ regardless of running them locally or on CI. We have at least the logic not to test modules which should not be affected by a change for PR validation purposes.

Would it be wiser to separate the tests out by their primary concern so that we can better handle them?

The question is to what purpose? There are multiple goals here:

Getting test results faster (or run more comprehensive tests in the same time) -> use faster dedicated runners
Run tests in a more difficult (slow) environment to find bugs that would otherwise not turn up -> run tests on slower hardware

I'm not against doing both, doing quicker (but happier path) testing in the default case but also having nightlies running in more constrained environments (only makes sense if someone has time to evaluate the results, this can be a fulltime job).
(I used to personally organize Jenkins runners for akka-http with the exact reasoning of getting more stable tests after having spent several weeks on doing nothing else than fixing flaky tests on slow hardware).
Also, the cost of dedicated hardware must be taken into account, not sure if we can get GHA usage summaries but at the current usage for pekko-core it might not be feasible to have dedicated runners, certainly not on cloud machines where I would expect the usage to be easily hundreds of dollars per month for the current usage.


Better support for GitHub pages

Feature Description
When you create a Github pages “project site” based on one of your repos, the url is http(s)://<username>.github.io/<repository> or http(s)://<organization>.github.io/<repository>
You can host an Evidence project with GitHub pages, but clicking Home will return a 404 because it will take you to http(s)://<organization>.github.io rather than http(s)://<organization>.github.io/<repository>
There should be a way to specify/adjust the root of the Evidence project so the hosted site works as expected.

Closing in favour of #603


Adding derive for FromPrimitive to enums.

Hey Keith, thanks for that!
Right now the main branch has got lot of breaking changes ready to be shipped as 0.19.0.
If you want this to be available as 0.18.3, I'd recommend creating an additional PR on that branch.

Thanks for the heads up! I'll add that as well.


Infinity loop in jsx file

Simple repro
<span>{a ?? ''}</span>;

/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;
/^(\d\d)$/;

thanks for the heads up, and sorry for the late reply!!! I'll look into this ASAP


App Settings: Generated UI - single text input

Requires #2139
The goal of this issue is to be able to define a single text-based setting for an app as per #2139, and have this result in a working text input on that app's setting page (and not the setting page of any other apps).

The outputted UI needs to contain the label & help text defined in the app's settings, and a standard text input pre-populated with the value from the app_settings table.
When saving the setting, it needs to run any validations (such as max length) defined in the settings, and save the new value.
Notifications for success, validation error, or other errors need to be generated such that they are consistent with the rest of Ghost.
The key part of completing the goal of this issue is creating a structure for defining settings UIs that can easily be output, so that as we expand the abilities of apps and this system, it will be able to generate screens with many and various settings.
Data binding will be a serious consideration for this issue.

Temporarily closing all of the app specific issues (tagged with later). Will review & reopen those that are still relevant when we kick off that project, which should be shortly after Zelda lands.


Bump OpenMMTools to 0.13.4

We're just running into travis timeouts with osx, so merging.
@lnaden: Do you have merge rights here?

Yes, I do, I got sidetracked today


content: Handle  horizontal lines

Final Result:

Fixes: #353

Thanks @chrisbobbe! I have restructured the code accordingly.


x/sys/unix: request for Ioctl*Int32 functions

One very relevant usage of ioctl functions (at least on Linux) is for querying extended attributes in files. As seen in https://man7.org/linux/man-pages/man2/ioctl_iflags.2.html ,

The type of the argument given to the FS_IOC_GETFLAGS and
FS_IOC_SETFLAGS operations is int *, notwithstanding the
implication in the kernel source file include/uapi/linux/fs.h
that the argument is long *.

What this means is that the attr field in  ioctl(fd, FS_IOC_GETFLAGS, &attr); and ioctl(fd, FS_IOC_SETFLAGS, &attr); is always a 32-bit signed integer. This means that using the IoctlGetInt and IoctlSetPointerInt functions is wrong for these values, because Go's int will be 32-bit or 64-bit, depending on the platform's pointer width. It happens to work on little endian platforms, but it will definitely be wrong on 64-bit big endian platforms (so at least ppc64 and some mips, maybe aarch64be? not sure go supports that).
x/sys/unix does provide IoctlGetUint32, but that has the wrong sign, unfortunately.
There is a workaround, which is using IoctlGetUint32 and converting to an int32, and using IoctlSetInt(fd, req, int(uintptr(unsafe.Pointer(attr))) instead of IoctlSetPointerInt, but these are hardly ergonomic.
Therefore, I would suggest providing IoctlGetInt32, IoctlSetPointerInt32 and IoctlSetInt32 functions. If homogenous coverage is desirable, a *Uint32 set could be added as well.

CC @ianlancetaylor @tklauser @bradfitz via https://dev.golang.org/owners

Adding IoctlGetInt32, IoctlSetPointerInt32 and IoctlSetInt32 sounds reasonable to me. Want to send a patch?
If I remember correctly, IoctlGetUint32 was added for IOCTL_VM_SOCKETS_GET_LOCAL_CID about which vsock(7) says: "Get the CID of the local machine.  The argument is a pointer to an unsigned int." Do you know of any relevant ioctl variants using unsigned int which would benefit from IoctlSetPointerUint32 and IoctlSetUint32 being added?

Adding IoctlGetInt32, IoctlSetPointerInt32 and IoctlSetInt32 sounds reasonable to me. Want to send a patch?

Sure, will try to whip one up!

Do you know of any relevant ioctl variants using unsigned int which would benefit from IoctlSetPointerUint32 and IoctlSetUint32 being added?

I didn't, which is why I suggested them more for symmetry than anything else. Some investigation gives me at least:
int ioctl(int fd, FAT_IOCTL_GET_ATTRIBUTES, uint32_t *attr);
int ioctl(int fd, FAT_IOCTL_SET_ATTRIBUTES, uint32_t *attr);
int ioctl(int fd, FAT_IOCTL_GET_VOLUME_ID, uint32_t *id);

which justify IoctlSetPointerUint32.
I have now realized that I'm not sure IoctlSetInt32 and IoctlSetUint32 make sense by themselves, since they will necessarily have to be simple casts from arg to uintptr(arg). Maybe it makes more sense to only add a IoctlSetUint function.

Relevant dicussion: https://github.com/golang/go/issues/14873 (https://go-review.googlesource.com/c/sys/+/26650/)

I started looking into implementing these, and it turns out that since 0cf1ed9e522b7dbb416f080a5c8003de9b702bf4 the IoctlSetPointerInt method silently truncates its argument to an int32, which would, funnily enough, make it adequate for my usage. @ianlancetaylor pointed out that it should have been named IoctlSetPointerInt32 (though the final function signature was a better fit), but I don't understand why it was dropped in https://go-review.googlesource.com/c/sys/+/150321
Should I open a separate issue to improve docs for this function and/or change its signature? IMO its value argument should already be int32 or it should check if value fits in the max range of int32 and error out otherwise (or it should be deprecated and IoctlSetPointerInt32 should become the correct function?).

Hmmm... in #46060 a proposal for IoctlSetIntPtr was made, but this int is just a bare int, without any indication whether it's int32 or int64. Should that be IoctlSetInt32Ptr and IoctlSetInt64Ptr?
For my use case, I'm compiling against a 32 bit system, so it's a non-issue and I can safely assure that ints are 32-bits, but I think problems may arise with it just being an int. I've also found some kernel ioctls with __kernel_loff_t as an arg) which is long long.
Maybe we should admit that not exposing the raw ioctl call is a mistake now? I've occasionally fell back to writing a syscall ioctl helper function because this package doesn't expose it && I couldn't find a compatible ioctl function signature.

The names of the ioctl functions refer to C, so I think it's OK if IoctlSetIntPtr refers to a 32-bit integer type.
Ah, I see the problem: the Go type is int, and people won't realize that this doesn't correspond to the C type.  That is a problem.  I think we can change the Go type to int32, but happy to hear other opinions.

Rather annoyingly we might four functions: IoctlSet(Ui|I)nt(32|64), which I think is also the case here?
I can see this bloat just continue increasing however. A quick search of
SYS_IOCTL for the Go language on Github gives almost 30k results: https://github.com/search?l=Go&q=SYS_IOCTL&type=code
A lot of these are from vendored/library code, but I'm sure there's a sizeable amount of code written that directly uses the ioctl syscall due to the limitations of the types offered in this package.
ioctls are a hot mess, and if the goal in this package is to try to support every single type, we'd end up playing catch-up with the ever increasing ioctl mess in the kernel.
Adding helpers for the (Ui|I)nt(32|64) types are sensible as most ioctls use this, but I don't know if it's a good idea to let this package continue playing catch up.

Addendum:
I dug deeper into kernel code, and I think we'd actually need them for all of
int8  int16  int32  int64
uint8 uint16 uint32 uint64

thus basically every int type with a specified width.
Some examples:
PPRSTATUS: uint8
RIO_CM_CHAN_CREATE: uint16
TIOCGPTN: uint32
ACPI_THERMAL_GET_TRT_LEN: uint64

TIOCSTI: int8
<NOT FOUND BUT NOT SURE IF PRESENT>: int16
TIOCSPTLCK: int32
FIOQSIZE: int64

You don’t actually need both signed and unsigned types as converting between them explicitly keeps the same bit pattern. All the kernel knows is the size of the argument. They might be useful to have for clarity though.


BREAKING: Make all APIs synchronous

Blocked by #154.
All of the APIs we use from @noble/curves are synchronous, meaning that we can make all functions in this package synchronous as well. This is a breaking change, because it changes the function signature for most functions.

Synchronous APIs:
yarn test  61.28s user 3.85s system 212% cpu 30.655 total

Asynchronous APIs:
yarn test  75.17s user 4.44s system 245% cpu 32.381 total


Default file logging

At the moment in the 4.0 branch (and also in the 3.0 branch), by default there is no file logging setup. Currently the framework/_config/logging.yml file is setup to use just the SilverStripe\Logging\HTTPOutputHandler which would mean any errors logged in this module just display in the browser.
It would be good to potentially add a file logging handler that is added to the logger interface to write any errors to a file (which could default to syslog for example), to ensure all errors are logged by default.

Hrm, I'm not sure this is something that the module should configure - instead, we should leave this to either platform-specific code (e.g. CWP includes this logging config), or website-specific code (e.g. to forward errors to a error monitoring service like Raygun.io or Papertrail.
@robbieaverill thoughts?

I think that so long as there is logging information generated by this module, then it's up to the configuration of the project on how to deal with that information.
I inspected and thought about closing this earlier on the same basis you raise @madmatt - but what I'm unsure of is whether or not this module generates anything useful to log. If not then at least that should be addressed.

If I understand correctly, the problem is that errors logged from this module are displayed in the browser, or swallowed up by the SilverStripe error handler.
I think the problem here is a manifestation of https://github.com/silverstripe/silverstripe-framework/issues/8044, which has an open pull request to separate the default log handler from the core SilverStripe error handlers. I'm going to close this for now, because fixing that issue will also fix this issue.

The most important part of the module to have logging has it, which is the SAML response parsing (https://github.com/silverstripe/silverstripe-saml/blob/master/src/Control/SAMLController.php#L97) so I'm happy with this being closed now that the module does actually log that info. Feel free to re-open if there's specific stuff that's missing from the logs @timkung.


Introduce ApplicationInsightsLoggerOptions to control logger behaviour

https://github.com/Microsoft/ApplicationInsights-aspnetcore/pull/574
https://github.com/Microsoft/ApplicationInsights-aspnetcore/pull/570
Both the above introduced this already,
Version Info
SDK Version :
.NET Version :
How Application was onboarded with SDK(VisualStudio/StatusMonitor/Azure Extension) :
OS :
Hosting Info (IIS/Azure WebApps/ etc) :

fixed with #570


CosmWasm setup for research  team

Description
Researches need simple  script to run test with several contracts loaded and executed. They need ability to obtain output of tests run in release mode.
Solution

document how to install proper version of rust and other dependencies if any (clang, protoc, wasm target etc)
add several wasm contracts into repository
add one tests which uploads, initializes and executes at least one instruction per contract with success
provide bash command to run this test in release mode
add trace level log output into SimpleWasmiVM  in memory usage methods (db_write, db_remove, db_read, etc) and other host functions.
provide simple bash command command to run test with all output showing access to storage from these contracts

Acceptance
Researches use guide to setup their environment and run tests in described mode. Any help to follow guide is provided to them is provided on request via medium they prefer.

I suggest to split it into 2 PR. One is simple doc without too many tests and tracing details. And than with all tracing and more contracts. I am not expert in wasm, so my estimate is is 2 days to get this done. @krl are you ok with that? No nix, no CI, just simple local Rust and bash.

https://github.com/ComposableFi/cosmwasm-vm/issues/28

https://docs.google.com/document/d/1uWYEEVqxQGMGz0p2NqFdYIuFngBiP5STqog_FwL1O6I/edit?pli=1#

https://docs.google.com/document/d/1uWYEEVqxQGMGz0p2NqFdYIuFngBiP5STqog_FwL1O6I/edit?pli=1&disco=AAAAwjhi4vI


Minor fixes for failing unit tests

Removed the Random.cls-meta.xml file from force-app/main/default/classes since there was no associated .cls file.
ErrorMailReceiver.cls and SetupRetryUnparseable_Test didn't set the required field dtm_RecordedDate__c so that was added.
Latest version of fflib_SecurityUtilsTest was pulled from the fflib github repo to fix issues where orgs created after Spring '21 don't have the Read Only profile anymore.

Can you please check if your setting the recorded date is not redundant to what I do in ParseErrors class.


save_all_weights and load_all_weights

Hey guys,
This is mostly just adapted from save_model and load_model in current version of Keras master model.py. Saves optimizer weights and model weights but not model configuration. Loads optimizer weights and model weights but you have to instantiate architecture yourself.
Functionally, save_all_weights goes halfway between save and save_weights.
I know this is redundant with Keras but way faster to implement here.
Please let me know your thoughts. This is my go-to method if I have something complicated that doesn't serialize well.
Cheers

@tboquet current version has the docstring fixes, test fixes, and removes the gitkeep.
Cheers

@farizrahman4u please take a quick look if you get a chance. This is mostly just a copy of the save/load code from keras removing certain lines. This loads model and optimizer weights into an existing model. Keras can only load model weights into an existing model or use model and optimizer weights to instantiate a new model.
I don't like the way this code is structured. However, it mirrors keras internals exactly, so if keras has a change or a bugfix we can mirror it easily.
Cheers


Extend K8s tool to add api headers i.e User impersonation

Hi team,
I couldn't find anything related to my inquiry in the issues, but I wanted to know if there is something on the road map to add kubectl api headers to extend kubectl commands through the vscode tool. Our team sometimes need to do something like kubectl --as <name-of-elevator-user> get secrets depending on the kubectl command and I thought it'd be really cool if the vscode tool could add some additional options to add certain headers to make the commands similar to the manual process.
I apologize if I didn't word this very well. this is my first time writing a post like this.  I'm happy to contribute if this at all is reasonable.

Hey @GinoSahagun thanks for opening this. We would need to update the code a bit but i think it could be a nice enhancement. How do you think this should work? Having an action similar to the incognito mode in the browser so that when switched all commands are executed by impersonating another user/group?
@itowlson @squillace WDYT?

Yes...  I like that suggestion it makes sense and it seems to align how kubernetes describes the "impersonation mode" and how users / groups can be used.

Sorry for the belated reply.  Impersonation mode sounds like the most practical way in the current design; the only other option I can think of is to have different tree entries you could switch between ("My Cluster" / "My Cluster As Seen By Keith").
This seems like it could have the potential for confusion if someone does non-k8s stuff for a while and forgets they're in impersonation mode.  What UI can/should we provide to remind them?  Browsers tend to change their chrome for private mode but we can't really mess with VS Code that way...!

Sorry for my belated reply. is it possible to have the present being similar to how clusters are shown, but have the user be shown in a similar structure and using a tab maybe or something?


Try to reduce bazel build time and binary size

Addresses https://github.com/google/heir/issues/847
This didn't reduce the build time all that much (30 minutes -> 25 minutes on my machine), but it does reduce the build time of heir-opt alone, and makes heir-opt 61 MiB when built with optimizations, vs the current 161 MiB

Copybara migration failed, patching internally.


[BUG] Bad Index Behavior after indexing an expanded tensordict

Describe the bug
Hello tensordict team, I'm unsure where this bug comes from, this happened after I called TensorDict.expand() on the tensordict and indexed it with a 2d tensor.
To Reproduce
Steps to reproduce the behavior.
Please try to provide a minimal example to reproduce the bug. Error messages and stack traces are also helpful.
Please use the markdown code blocks for both code and stack traces.
import torch
import tensordict

a = torch.arange(9).reshape(3,3)
d_1 = tensordict.TensorDict({
    "a": a,
}, batch_size=(3,3))
d_2 = d_1.unsqueeze(0).expand(1000, 3, 3)
idx = torch.arange(3).unsqueeze(0).expand(3,3) # construct a 2d array to index into d_2
d_2_1 = d_2[idx] # this works
print(d_2_1)
d_2_2 = d_2_1[0:] # this errors

Traceback (most recent call last):
    File "/home/quantumcookie/miniconda3/envs/orangerl/lib/python3.8/site-packages/tensordict/base.py", line 264, in __getitem__
    return self._index_tensordict(index)
  File "/home/quantumcookie/miniconda3/envs/orangerl/lib/python3.8/site-packages/tensordict/_td.py", line 773, in _index_tensordict
    names = self._get_names_idx(index)
  File "/home/quantumcookie/miniconda3/envs/orangerl/lib/python3.8/site-packages/tensordict/_td.py", line 1306, in _get_names_idx
    names = [names[i] if i is not None else None for i in idx_to_take]
  File "/home/quantumcookie/miniconda3/envs/orangerl/lib/python3.8/site-packages/tensordict/_td.py", line 1306, in <listcomp>
    names = [names[i] if i is not None else None for i in idx_to_take]

Expected behavior
This should index completely fine
System info
torch==2.2.0
tensordict==0.3.0
Additional context
Add any other context about the problem here.
Reason and Possible fixes
Unsure
Checklist

[x] I have checked that there is no similar issue in the repo (required)
[x] I have read the documentation (required)
[x] I have provided a minimal working example to reproduce the bug (required)

Some additional info:
I installed tensordict==0.2.1 instead and it looks like this code would work fine with that version

Thanks for reporting ! I'll look into it


[Core] Add GetRandomFloat to API

I've had to implement this function for my use cases many times in the past and I think it would be beneficial to add it along side GetRandomValue. I also went ahead and updated the GetRandomValue example to include an example of the GetRandomFloat in use.

@REDl3east I prefer to avoid adding this function, it can be implemented on user side with provided ones.


[multistage] refactor: use DataTable interface where possible (MINOR)

this is a MINOR PR, mostly the result of automatic refactoring tools from IntelliJ. this will help set up an additional refactor as part of #9711 where we will split MetadataBlock into two separate blocks that don't implement BaseDataBlock: MetadataBlock and SignalBlock, both of which will have their data encoded in JSON instead of the custom pinot format.

Codecov Report

Merging #9721 (7089398) into master (229c55e) will increase coverage by 4.56%.
The diff coverage is 83.33%.

@@             Coverage Diff              @@
##             master    #9721      +/-   ##
============================================
+ Coverage     63.02%   67.58%   +4.56%
+ Complexity     5229     5176      -53
============================================
  Files          1938     1448     -490
  Lines        104130    75810   -28320
  Branches      15780    12067    -3713
============================================
- Hits          65625    51237   -14388
+ Misses        33640    20915   -12725
+ Partials       4865     3658    -1207

Flag
Coverage Δ

integration1
?

integration2
?

unittests1
67.58% <83.33%> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...g/apache/pinot/common/datablock/BaseDataBlock.java
80.35% <ø> (+0.22%)
:arrow_up:

...g/apache/pinot/common/datatable/BaseDataTable.java
78.35% <0.00%> (-0.59%)
:arrow_down:

...a/org/apache/pinot/common/datatable/DataTable.java
92.06% <ø> (ø)

...apache/pinot/query/mailbox/GrpcSendingMailbox.java
90.90% <ø> (ø)

...ache/pinot/common/datablock/ColumnarDataBlock.java
60.00% <50.00%> (+1.37%)
:arrow_up:

...t/query/runtime/blocks/TransferableBlockUtils.java
90.47% <50.00%> (ø)

...a/org/apache/pinot/common/datablock/BlockType.java
70.00% <70.00%> (ø)

.../pinot/core/common/datablock/DataBlockBuilder.java
72.98% <80.00%> (ø)

.../apache/pinot/common/datablock/DataBlockUtils.java
85.98% <100.00%> (ø)

...g/apache/pinot/common/datablock/MetadataBlock.java
66.66% <100.00%> (+3.03%)
:arrow_up:

... and 759 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

^ should be covered by #9731 ?


Update README.md

Corrected typo UDT => UDP

Hey @frank-dspeed,
Thanks but the protocol is correct UDT :)
more details: https://en.wikipedia.org/wiki/UDP-based_Data_Transfer_Protocol)


What dtlane vector maps contains?

What is the 'Dir', 'Apara','r','slope','cant' in dtlane vector maps and their significance?

Thanks for using Autoware and for your question. However the issue tracker is for posting confirmed bugs and feature requests.
We ask that you please ask questions on the Autoware Discourse category following our support guidelines: https://github.com/CPFL/Autoware/wiki/Support-guidelines. Please pay particular attention to the information we ask you to provide.


Limit the length of the invalid rule indices message

This closes #1975, which shows that extremely long invalid rule indices errors, as shown below, throw an error if the message is long enough.

This PR limits the length of the invalid indices string to 50 characters to avoid errors like the one shown in Sentry.

Changed the placeholder since it's ... everywhere else across the code.


x/tools/gopls: use cases for build tags

What use cases do users have for build tags? Please post your own examples.
To start:

OS-specific
Certain package trees need specific build tags (e.g. WASM)
Some companies use build tags for things like integration testing, different environments etc.
+build ignore file main files

I have used build tags in a library so that it only depends on stdlib by default but usage of 3rd party libraries like github.com/pkg/errors can be enabled via build tags.

In TinyGo (which should eventually support IDEs), we use build tags to select various things:

chip vendor (stm32, nrf), chip (stm32f103xx, nrf52840), and board names (bluepill, pca10056)
which GC style is being used (currently gc.none, gc.leaking, and gc.conservative)

More uses may be added in the future.

Build tags are commonly used for go generate - so that generators can be stored alongside the package in which they are used

We have used build tags mainly in two places:

OS specific system calls
New Go language or standard library features (e.g. // +build go1.13 in one file that implements something using features/code only available in 1.13, and // +build !go1.13 in the other file with the 1.12 implementation).

That being said, we don't use build tags very much.

Two files, named something like paranoia.go and nop_paranoia.go, which both define the same const bool, one setting it to true and one to false. Then elsewhere in the code:
if paranoia {
        // extra testing
}

This way, we can run with the extra testing sometimes, but omit expensive tests entirely at compile time. (Some of the tests are inside hot loops, so being able to get rid of even the cost of the branch is actually relevant.)
Have also used them for what would typically be #defines in C; functionality changes which are innately program-wide and must be resolved at compile time because array sizes (for instance) depend on them.
I've seen a package (hajimehoshi/ebiten) ship with a number of examples in subdirectories, marked with +build example so they don't get built by default but are available for users to experiment with.

I used build tags to add enhanced debugging that goes into tight loops. Using an if statement much like @seebs .
One file id debug.go and the other debug_flg.go
They each define a debug constant.
then if I need to debug tight loops I'll eighter flip the const in debug.go from false to true (this is what I've mostly been doing this, because usually just want a single package) or build with the debug flag, which will cause debug_flg to compile and not debug.go to compile.
This being said, this is probably not the best way to do this.

In google.golang.org/protobuf:

// +build ignore to exclude an expensive integration test from go test ./....
// +build purego to select a slower implementation of various operations that doesn't depend on the unsafe package.
// +build  go1.12 to use (*reflect.Value).MapRange when available.
// +build protoreflect to disable fast-path implementations of various operations to allow testing the reflection-based fallbacks.
// +build proto1_legacy to enable support for some ancient features nobody should use. Used so we can run tests for that support.

We use build tags to separate the almost identical input and output files of
our code generator.
Specifically, we have an input file that is valid Go code, and has a build tag
to exclude it from the build.
$ cat foo.go
// +build magic
package foo

func Bar() {
    ...
}

The code generator takes this file and spits out a nearly identical file with
the build tag inverted and some changes to the function bodies.
$ magic ./...
$ cat foo_gen.go
// +build !magic
package foo

func Bar() {
    ...
}

The code generator (magic) only considers files with the magic build tag.
This excludes the generated foo_gen.go from consideration when the tool is
run again. The go tool is invoked without the build tag so only the
generated file is compiled in the final binary.

magic
go build
// +build

foo.go
Use
Ignore
magic

foo_gen.go
Ignore
Use
!magic

I use build tags to separate integration tests and examples (depending on external resources such as database) from unit tests (no external dependencies).

// +build integration
// +build example
so go test won't run them by default

In the Tegola project  we also use it select which features are enabled or not.

atlas/cache_s3.go:// +build !noS3Cache
atlas/provider_gpkg.go:// +build !noGpkgProvider
atlas/cache_redis.go:// +build !noRedisCache
atlas/cache_azblob.go:// +build !noAzblobCache
atlas/provider_postgis.go:// +build !noPostgisProvider
cmd/tegola/pprof.go:// +build pprof
provider/gpkg/gpkg.go:// +build cgo
server/viewer_disabled.go:// +build noViewer
server/viewer.go:// +build !noViewer

for building part of a program as a seperate binary. eg:part of a program parses some file, this would be used as an API by default, but with an alternate main file you can use the same codebase to build a parser(or any other modular thing) that would output to stdout or another file.
I used something similar in a "bigger" project where the program also had to use some modules(not go modules) as external binary and communicate over std in/out.
We could have a main wrapper that would usually not build, but with build tags we could build specific modules and use them in testing(or for example as seperate downloads)

The gioui.org/ui/app package is like package os, but for GUI programs: it uses build tags to distinguish between GOOS. Some examples:
// +build !js
// +build android
// +build darwin,ios
// +build linux windows

I also use the standard GOOS build tags in file names: os_js.go, gl_android.go, egl_windows.go etc.

Just to expand on the WASM example from my persective. The typical setup is that a subpackage/subtree of packages within a module (a module that has both backend and frontend code) has a preferred build configuration, specifically GOOS=js GOARCH=wasm. The key here being that whilst within the module, GOOS=js GOARCH=wasm is one such build, for this subpackage/subtree of packages, GOOS=js GOARCH=wasm is the default/preferred. i.e. when editing a file in the subpackage/subtree of packages, the implied build constraint is GOOS=js GOARCH=wasm.

Adding a separate comment about where this support for understanding the universe of build configurations for a module is required:

in editor; the UI/UX of how to choose the build is perhaps one of the trickiest points. @eliasnaur would prefer (in the context of his setup) automatic "selection" of the relevant build based on the file being edited. It's unclear whether this will work in all situations but sounds like an obviously beneficial and sensible default
(static analysis) tools, e.g. staticcheck (cc @dominikh) and apidiff (cc @jba)
CI systems
...

This list is likely incomplete

The wire dependency injection tool uses
// +build wireinject

to prevent Go code used for configuration from being included in the actual build.
More at https://github.com/google/wire/blob/master/docs/guide.md and from @zombiezen.

Use of // +build tools to add tool dependencies to go.mod, as suggested in How can I track tool dependencies for a module?.

@breml - that's a particularly interesting example because such a file will never compile.

I have used build tags to enforce that a package is being built with a particular version of Go: https://github.com/zombiezen/gg/blob/13b5a059c5addfa7543b2b572650d9544bfcb729/internal/git/force_go19.go This use case is likely obsolete now that the go.mod file includes language version information, but such code may exist out in the wild.
Anecdotally, I most often use build tag (or the filename equivalent thereof) mechanisms in order to provide Unix-y and Windows equivalent functions.
Happy to answer specifics about Wire's usage of build tags if needed, but the docs are a pretty good place to start understanding. From the perspective of gopls-like static analysis, it should be pretty similar to the OS-specific code use case in that the same symbols get redefined.

Thanks all for your detailed replies! I will be working on compiling your answers into a document, and hopefully soon we'll be able to make some progress on supporting build tags in gopls.

I teach and use Go for programming labs, and keep solutions and skeleton code for the students in the same folder to ensure consistency with common files and to remember to change both if necessary.
I use // +build solution in solution files and // +build !solution in skeleton code, since they have overlapping type/func definitions. (This of course results in numerous compile errors when using gopls in VSCode.)
I run tests using go test -tags solution.
Running go test without the solution tag should fail the tests.
I then copy the skeleton code to a separate repo to share with the students.

Build tags are crucial for projects that have open source and enterprise variants.

gobbi, a package that provides bindings to GTK GUI toolkit, uses Go build tags to allow targeting of specific library versions. See here for more details.

I have one file with '// +build debug', and an almost identical file in the same folder with '// +build !debug'. gopls flagged every constant, variable and function as being redefined between the two files.  The debug file just prints log messages to replace functions that make major changes to the running state of the system.


Eliminate redundancies in StateReader

We are trying to integrate starknet_in_rust in to our full node implementation Juno. But there are some redundancies and not really useful methods that could force the underlying storage to be refactored with no obvious benefit for the user.
https://github.com/lambdaclass/starknet_in_rust/blob/9029fa1db6edf7f31bd719bbb6b149da66f094e9/src/business_logic/state/state_api.rs#L12-L46

get_contract_class can be completely replaced with get_compiled_class_by_class_hash
get_compiled_class is not really useful, because compiled class hashes are not used in execution. Contract are deployed by their class hashes (ie get_class_hash_at returns class hashes) and VM should be able to just use class hash of a Sierra class to be able to fetch the compiled class without the intermediate step of fetching the compiled class hash. Which eliminates the need for get_compiled_class_hash as well.

What we believe/proposing the StateReader to look more like;
pub trait StateReader {
    /// Returns the contract class of the given class hash.
    fn get_contract_class(&mut self, class_hash: &ClassHash) -> Result<CompiledClass, StateError>;
    /// Returns the class hash of the contract class at the given address.
    fn get_class_hash_at(&mut self, contract_address: &Address) -> Result<ClassHash, StateError>;
    /// Returns the nonce of the given contract instance.
    fn get_nonce_at(&mut self, contract_address: &Address) -> Result<Felt252, StateError>;
    /// Returns the storage value under the given key in the given contract instance.
    fn get_storage_at(&mut self, storage_entry: &StorageEntry) -> Result<Felt252, StateError>;
    /// Counts the amount of modified contracts and the updates to the storage
    fn count_actual_storage_changes(&mut self) -> (usize, usize);
}

Hi! Yeah we noticed that there are duplication in the API so we are currently working on refactoring this

Thanks, happy to help in any way.

I'd also like to point out that it'd be great to address handling of missing values. Like: get_class_hash_at() is now expected to return 0 for non-existent contract addresses. This seems to be an antipattern: it'd be much better if the return type would be an Option if calling the method with a non-existent address is not an error.


update docs/api: yield take(...) return an action object

Q
A

Fixed Issues?
fixes #1552

Patch: Bug Fix?
only update docs

Major: Breaking Change?
no

Minor: New Feature?
no

Tests Added + Pass?
No + Yes

Any Dependency Changes?
No

Describe what yielding take returns, details in the issue.

Thanks!


client: Handle untimeout_success, Fixes #247

Fixes #247

I'll see about submitting a PR to the docs project as well.


Carthage build fails because of "no shared framework schemes"

**Have read the known issues? **
[X] Yes, I have read and am familiar with the list of known issues.
**Have tested on a physical device? **
[N/A - can't build ] Yes, I have tested on physical iOS device and the reported issue is still present.
**Does code retain the Connectivity object? **
[ N/A - can't build] Yes, my implementation has a strong reference to the Connectivity object so that it is not deallocated during use.
Describe the bug
Carthage fails to build the code. Error:
*** Skipped building Connectivity due to the error:
Dependency "Connectivity" has no shared framework schemes for any of the platforms: iOS
To Reproduce
Steps to reproduce the behavior:
Create a new Xcode project (Xcode 11.4.1)
Add Cartfile entry:
github "rwbutler/Connectivity"
Run command: carthage update --platform iOS
Error result as above.
Carthage version is 0.3.4
I have tried referencing Connectivity v4 with same result
Expected behavior
Carthage should build the framework.
Screenshots
N/A
Desktop (please complete the following information):

OS: Catalina
Browser N/A
Version N/A

Smartphone (please complete the following information):
N/A
Additional context
N/A

I am also getting similar issue on xcode 12 while running carthage update --platform ios
I am using Connectivity version 1.3.2.
Skipped building Connectivity due to the error: Dependency "Connectivity" has no shared framework schemes for any of the platforms: iOS

I am also getting similar issue on xcode 12 while running carthage update --platform ios
I am using Connectivity version 1.3.2.
Skipped building Connectivity due to the error: Dependency "Connectivity" has no shared framework schemes for any of the platforms: iOSI am also getting similar issue on xcode 12 while running carthage update --platform ios
I am using Connectivity version 1.3.2.
Skipped building Connectivity due to the error: Dependency "Connectivity" has no shared framework schemes for any of the platforms: iOS

@kumarGautamAramark Do you have any idea for a workaround?

Thanks for raising this issue @muiscatron - could try carthage update and see whether 5.0.1 works for you?

I think your 5.0.1 fix may have broken something. I just updated using Carthage and now get this error:
/Users/home/Documents/MyApp/MyApp.xcodeproj Building for iOS Simulator, but the linked framework 'Connectivity.framework' was built for iOS.

@inPhilly It sounds like when the framework was built it was compiled for iOS (physical device) but not for simulator for some reason. Could you provide me with a little more information regarding:

The version of Xcode you're using (if you have multiple versions you can find out which one is being used by typing Xcode-select -p)
The version of Carthage you're using
The command that was used to update the frameworks e.g. carthage update --platform iOS

Thanks in advance!

@muiscatron @inPhilly Hoping this is fixed now? Am going to close this issue for now as I haven't heard any further but happy to re-open if it turns out there's still a problem.


Cannot run pip install <package> with Python 3.4 images

Running pip install <package> fails with the following:
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.4/site-packages/py-1.8.1.dist-info'
Consider using the `--user` option or check the permissions.

Reproduction
docker run --rm circleci/python:3.4 pip install tox

Output:
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).
Collecting tox
  Downloading https://files.pythonhosted.org/packages/a7/0c/ed234b83d2c4fcef1cfccf97371183d51dafae62e64334de34d0a6333114/tox-3.14.0-py2.py3-none-any.whl (80kB)
Collecting filelock<4,>=3.0.0 (from tox)
  Downloading https://files.pythonhosted.org/packages/93/83/71a2ee6158bb9f39a90c0dea1637f81d5eef866e188e1971a1b1ab01a35a/filelock-3.0.12-py3-none-any.whl
Collecting toml>=0.9.4 (from tox)
  Downloading https://files.pythonhosted.org/packages/a2/12/ced7105d2de62fa7c8fb5fce92cc4ce66b57c95fb875e9318dba7f8c5db0/toml-0.10.0-py2.py3-none-any.whl
Collecting pluggy<1,>=0.12.0 (from tox)
  Downloading https://files.pythonhosted.org/packages/a0/28/85c7aa31b80d150b772fbe4a229487bc6644da9ccb7e427dd8cc60cb8a62/pluggy-0.13.1-py2.py3-none-any.whl
Collecting py<2,>=1.4.17 (from tox)
  Downloading https://files.pythonhosted.org/packages/99/8d/21e1767c009211a62a8e3067280bfce76e89c9f876180308515942304d2d/py-1.8.1-py2.py3-none-any.whl (83kB)
Requirement already satisfied: six<2,>=1.0.0 in /usr/local/lib/python3.4/site-packages (from tox) (1.12.0)
Collecting importlib-metadata<1,>=0.12; python_version < "3.8" (from tox)
  Downloading https://files.pythonhosted.org/packages/f6/d2/40b3fa882147719744e6aa50ac39cf7a22a913cbcba86a0371176c425a3b/importlib_metadata-0.23-py2.py3-none-any.whl
Collecting packaging>=14 (from tox)
  Downloading https://files.pythonhosted.org/packages/98/42/87c585dd3b113c775e65fd6b8d9d0a43abe1819c471d7af702d4e01e9b20/packaging-20.1-py2.py3-none-any.whl
Requirement already satisfied: virtualenv>=14.0.0 in /usr/local/lib/python3.4/site-packages (from tox) (16.4.3)
Collecting zipp>=0.5 (from importlib-metadata<1,>=0.12; python_version < "3.8"->tox)
  Downloading https://files.pythonhosted.org/packages/21/21/ff0bfa221d5f5c5dc495977219c4de8f44ba8203f6724fb648113f9902cb/zipp-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: pathlib2; python_version == "3.4.*" or python_version < "3" in /usr/local/lib/python3.4/site-packages (from importlib-metadata<1,>=0.12; python_version < "3.8"->tox) (2.3.3)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.4/site-packages (from packaging>=14->tox) (2.3.1)
Requirement already satisfied: scandir; python_version < "3.5" in /usr/local/lib/python3.4/site-packages (from pathlib2; python_version == "3.4.*" or python_version < "3"->importlib-metadata<1,>=0.12; python_version < "3.8"->tox) (1.10.0)
Installing collected packages: filelock, toml, zipp, importlib-metadata, pluggy, py, packaging, tox
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.4/site-packages/filelock.py'
Consider using the `--user` option or check the permissions.

You are using pip version 19.0.3, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Expected
Pip install the package without the need of sudo or --user.
Investigation
I have tried the above with circleci/python:3.4, circleci/python:3.4.9, and circleci/python:3.4.10 which all experience the same issue.
I have also tried with python:3.4 which does not have this issue.
If you look at the ownership of /usr/local/lib/python3.4 it is owned by root:staff.
$ circleci@e5c83fc0d329:/$ ls -hal /usr/local/lib/
total 11M
drwxrwsr-x 1 root staff 4.0K Mar 20  2019 .
drwxrwsr-x 1 root staff 4.0K Feb 28  2019 ..
lrwxrwxrwx 1 root staff   20 Mar 20  2019 libpython3.4m.so -> libpython3.4m.so.1.0
-r-xr-xr-x 1 root staff  11M Mar 20  2019 libpython3.4m.so.1.0
-r-xr-xr-x 1 root staff 7.5K Mar 20  2019 libpython3.so
drwxr-sr-x 2 root staff 4.0K Mar 20  2019 pkgconfig
drwxrwsr-x 4 root staff 4.0K Mar  5  2019 python2.7
drwxr-sr-x 1 root staff 4.0K Mar 20  2019 python3.4
drwxrwsr-x 3 root staff 4.0K Mar  5  2019 python3.5

The only thing I can really find that is different between say circleci/python:3.4 and circleci/python:3.5 is in 3.5 the owner of /usr/local/lib/python3.5 is root:root and not root:staff (like it is in circleci/python:3.4) and the version of pip is different.
For now I will try to just use python:3.4 since that does work for me.

Our Python v3.8 image will work without using sudo or --user. These differences between the versions occur from the upstream images. We build each one the same on our end.
You can avoid the discrepancy by just using sudo or --user in CircleCI always. Then every image will work. One of the reasons why this isn't needed with the upstream image is because that image runs as the root user already (an implied sudo basically). The CircleCI images run with the circleci user. Different user and permissions.
sudo or --user will ensure builds work.


Removing references to whisk object

Whisk object is deprecated and will soon be removed.  Updating code that uses whisk object.

Looks good LGTM

merging now


Changed the welcome screen text for deployments

Pull Request Test Coverage Report for Build 330248731

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 39.668%

Totals

Change from base Build 330209195:
0.01%

Covered Lines:
20608

Relevant Lines:
47690

💛  - Coveralls


Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll

INTEL MKL ERROR: The specified module could not be found. mkl_intel_thread.dll.
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll

Btw, I'm using AMD not Intel :thinking:

Solution: The repo used older version of dl4j. Once using the newer version, the program can be run properly. To change the version of dl4j or any other maven dependencies please follow the steps explained here:
https://github.com/CertifaiAI/CertifAI-Knowledge-Base/wiki/Creating-a-new-Maven-Project


Please add iFrame element contentWindow property

Please add iFrame element contentWindow property

From MDN:

From the DOM iframe element, scripts can get access to the window object of the included HTML page via the contentWindow property. The contentDocument property refers to the document element inside the iframe (this is equivalent to contentWindow.document), but is not supported by Internet Explorer versions before IE8.

Looks like this is now covered by MDN data:
https://caniuse.com/mdn-api_htmliframeelement_contentwindow
https://caniuse.com/mdn-api_htmliframeelement_contentdocument
(via mdn/browser-compat-data#2017)

Thank you. I close the issue.


python-package-init nixpkgs-pytools is missing checkInputs = [ pytest ]

when trying to package the nixpkgs-pytools with itself it seems to be missing pytest in checkInputs:
{ lib
, buildPythonPackage
, fetchPypi
, jinja2
, setuptools
}:

buildPythonPackage rec {
  pname = "nixpkgs-pytools";
  version = "1.0.1";

  src = fetchPypi {
    inherit pname version;
    sha256 = "0796c6e95daeb3d7e61c9c53126d95ba6a48f84b995b12b60f45619caf28a574";
  };

  propagatedBuildInputs = [
    jinja2
    setuptools
  ];

  meta = with lib; {
    description = "Tools for removing the tedious nature of creating nixpkgs derivations";
    homepage = https://github.com/nix-community/nixpkgs-pytools/;
    license = licenses.mit;
    # maintainers = [ maintainers. ];
  };

Looked into this and the issue is that pypi does not store all of the dependencies in the json api. So probably not worth using then? Take for example https://pypi.org/pypi/nixpkgs-pytools/json. So it is probably best to mock the package and not rely on the json api.

@costrouc how come it does sometimes work? Where is the information coming from normally?

So the "neat" non trivial part of nixpkgs-pytools is that is mocks setuptools/distutils setup(...) method. With this method it downloads the package and runs setup.py. I should probably do this by default. Originally I thought that using the api could speed things up.

i quite like the idea of mocking setup.py , i thought this is the default actually. Too bad the api does not respond with test requirements, probably because normally you do not need this info when installing the package.

This is now fixed in the latest master. Here is the generated code. Mocking setup(...) is now the default
{ lib
, buildPythonPackage
, fetchPypi
, pytest
, jinja2
, setuptools
}:

buildPythonPackage rec {
  pname = "nixpkgs-pytools";
  version = "1.0.1";

  src = fetchPypi {
    inherit pname version;
    sha256 = "0796c6e95daeb3d7e61c9c53126d95ba6a48f84b995b12b60f45619caf28a574";
  };

  checkInputs = [
    pytest
  ];

  propagatedBuildInputs = [
    jinja2
    setuptools
  ];

  meta = with lib; {
    description = "Tools for removing the tedious nature of creating nixpkgs derivations";
    homepage = https://github.com/nix-community/nixpkgs-pytools/;
    license = licenses.mit;
    # maintainers = [ maintainers. ];
  };
}


Refund Transaction Not Being Confirmed

Hi,
Thanks very much for the amazing work you are doing here to make swaps available.  The consequences can only be imagined.
For once in my life I decided to be cautious and exercise the failure option first.  To that end I created and initiated a contract (using ltcatomicswap) then waited the 48 hours for the refund to be available.
I submitted the refund transaction via https://live.blockcypher.com/ltc/pushtx/ but it still hasn't confirmed, and as far as I can tell it hasn't been relayed, other Litecoin block explorers can't find the transaction.
Publishing the refund transaction using litecoin 0.14.2 sendrawtransaction "failed: 64: version (code -26)"
( getinfo  returns in part: "version": 140200,
"protocolversion": 70015,
"walletversion": 60000)
The current situation is: https://live.blockcypher.com/ltc/address/3CH9sjVpApvQR31U8niMyrYg3MzR9466uS/
Suggestions to remedy the situation or help me understand what I've done wrong would be appreciated.
In case it's significant, the refund address used in the initiate below is from the wallet that the local litecoin instance controls.
ltcatomicswap -rpcuser user -rpcpass atomic initiate LiYeAuZHCYUDxe6zpRSthneYew98My7meB 0.5
warning: falling back to mempool relay fee policy
Secret:      [REDACTED]
Secret hash: 80add26acbcd57e618a77573e5fe1696e85d9793

Contract fee: 0.000224 LTC (0.00100000 LTC/kB)
Refund fee:   0.000281 LTC (0.00102182 LTC/kB)

Contract (ZNdPisrRRMg7kYTxyFhGMFxgFCKAT4VUKU):
63a61480add26acbcd57e618a77573e5fe1696e85d97938876a914ffd3b1bd0c5a94762f41b155697e2d267a28c57e67046b17cb59b17576a9145a238ba258c2ec0bc3076038d8f3f97cd65b095a6888ac

Contract transaction (9948ad4738aba10ba57171ff2703fd41bbb6e09fd058fdac95c9fa4d0c27e108):
0200000001bf922f652c3ab84d233840a010e5e38ddc175de9cd0aaf015d1271d759ac1d95000000006b4830450221008208604bd6119c65a2fa7ea4a268f0831129fda31f0dd22f90b5138fc4d55d8102206933d1ab5d8476f401e26be7aca63a34d6e63fb1eaf893cc6238170640fc8f02012103bc62efac4d28be76d073e21bd924ce2a294a7dada17cfab7c186925e3f576b8ffeffffff0282d8464f020000001976a914da4cd7aac092d0acb20488f816214778171e39e488ac80f0fa020000000017a914742609c49d65ae0ed34416284f56773ecef240078700000000

Refund transaction (35452de16c7856c97214bbfc5f64978dcd263180db35ceb9c85f5ef32b88a181):
000000000108e1270c4dfac995acfd58d09fe0b6bb41fd0327ff7171a50ba1ab3847ad489901000000be47304402205838b0907d8819277909355cfbe72642cc3501fc976b4f8d1aa71292071fabde0220105692802184ec8176052654b74dd4db438637c9609398e02931a884314808a901210371b18eeab61b58d22e3c92ef0826e619dfa110a588d7db312421085e71a7b97d004c5163a61480add26acbcd57e618a77573e5fe1696e85d97938876a914ffd3b1bd0c5a94762f41b155697e2d267a28c57e67046b17cb59b17576a9145a238ba258c2ec0bc3076038d8f3f97cd65b095a6888ac0000000001bc82fa02000000001976a914da4cd7aac092d0acb20488f816214778171e39e488ac6b17cb59

Publish contract transaction? [y/N] y
Published contract transaction (9948ad4738aba10ba57171ff2703fd41bbb6e09fd058fdac95c9fa4d0c27e108)

Thanks for testing and reporting.  Looks like an error with the refund transaction being created with the wrong tx version.  I'll get that fixed, and i think it would also be a good idea to add a separate refund step to the commands so that refunds can be created after the fact if necessary (if the refund tx was lost, if it was performed on a blockchain that still had malleable txs, etc.)

Thank you.
While I'm relieved to know the problem isn't of my creation, it would help to know if I can correct the issue (and recover my LTC) by adjusting the transaction?  Naively comparing the contract and refund transactions, it would seem 02 replacing the leading 00 might suffice?
Or could I continue and redeem using my knowledge of the secret?

You'll need to use redeem once it's ready.  Manually modifying the tx like that will cause the signatures validation to fail.  I just implemented the redeem command for the dcr tool, in process of testing it now, and then will port it to the rest.

If you used your own address as the redeemer's address in the contract, you could also spend it with the redeem command and the secret.  I'm still adding that refund command but that will allow you sooner access to those funds.

@HamishMacEwan give #24 a try if you haven't already redeemed the output.

I have attempted to redeem the contract (the advantage of being party A & B) but as far as I can tell without success.
Although auditcontract works:

ltcatomicswap auditcontract 63a61480add26acbcd57e618a77573e5fe1696e85d97938876a914ffd3b1bd0c5a94762f41b155697e2d267a28c57e67046b17cb59b17576a9145a238ba258c2ec0bc3076038d8f3f97cd65b095a6888ac<PHONE_NUMBER>bf922f652c3ab84d233840a010e5e38ddc175de9cd0aaf015d1271d759ac1d95000000006b4830450221008208604bd6119c65a2fa7ea4a268f0831129fda31f0dd22f90b5138fc4d55d8102206933d1ab5d8476f401e26be7aca63a34d6e63fb1eaf893cc6238170640fc8f02012103bc62efac4d28be76d073e21bd924ce2a294a7dada17cfab7c186925e3f576b8ffeffffff0282d8464f020000001976a914da4cd7aac092d0acb20488f816214778171e39e488ac80f0fa020000000017a914742609c49d65ae0ed34416284f56773ecef240078700000000
Contract address:        ZNdPisrRRMg7kYTxyFhGMFxgFCKAT4VUKU
Contract value:          0.5 LTC
Recipient address:       LiYeAuZHCYUDxe6zpRSthneYew98My7meB
Author's refund address: LTSZfDdjmtURi35Auc7eHBpHH4aCdADPr8

Secret hash: 80add26acbcd57e618a77573e5fe1696e85d9793

Locktime: 2017-09-27 03:13:47 +0000 UTC
Contract refund time lock has expired

redeem (same paramters plus secret):

ltcatomicswap redeem 63a61480add26acbcd57e618a77573e5fe1696e85d97938876a914ffd3b1bd0c5a94762f41b155697e2d267a28c57e67046b17cb59b17576a9145a238ba258c2ec0bc3076038d8f3f97cd65b095a6888ac<PHONE_NUMBER>bf922f652c3ab84d233840a010e5e38ddc175de9cd0aaf015d1271d759ac1d95000000006b4830450221008208604bd6119c65a2fa7ea4a268f0831129fda31f0dd22f90b5138fc4d55d8102206933d1ab5d8476f401e26be7aca63a34d6e63fb1eaf893cc6238170640fc8f02012103bc62efac4d28be76d073e21bd924ce2a294a7dada17cfab7c186925e3f576b8ffeffffff0282d8464f020000001976a914da4cd7aac092d0acb20488f816214778171e39e488ac80f0fa020000000017a914742609c49d65ae0ed34416284f56773ecef240078700000000 [REDACTED `secret`]

Complains:
getrawchangeaddres: status code: 401, response: ""
Thanks for your suggestion, but #24 is a bit beyond my git-ability.  I can just manage to git pull origin master but no more.

That 401 error indicates an authentication error (your rpc user and password weren't set or didn't match).

DOH
Given that oversight, I think I'd be pushing my luck with anything that advanced.  You wouldn't believe how many times I used walletpassphrase to fix that 401...

Blockcypher is having a cow:   DOUBLE SPEND ATTEMPT!
https://live.blockcypher.com/ltc/address/3CH9sjVpApvQR31U8niMyrYg3MzR9466uS/
But things are looking good.
This won't be my last experiment with this technique, so I'll get chance to try #24  Thanks @jrick!

Ok, so I got brave and after completely losing my way somehow ended up with a working 6 option ltcatomicswap and a new and improved refund transaction.
020000000108e1270c4dfac995acfd58d09fe0b6bb41fd0327ff7171a50ba1ab3847ad489901000000be473044022037e68d167b538636e8d9e7cbf8293e0947533f6eac7c81194be1993475edeb88022068e928955e8e9c6bcf383164f68373d2c885f9bfdba46bba9785db25d42a1c5601210371b18eeab61b58d22e3c92ef0826e619dfa110a588d7db312421085e71a7b97d004c5163a61480add26acbcd57e618a77573e5fe1696e85d97938876a914ffd3b1bd0c5a94762f41b155697e2d267a28c57e67046b17cb59b17576a9145a238ba258c2ec0bc3076038d8f3f97cd65b095a6888ac0000000001bc82fa02000000001976a91468a1a081207bb54a63bb8accb0bcfe8bc9a188e088ac6b17cb59

sendrawtransaction quite properly complained 18: bad-txns-inputs-spent (code -26) but clearly the tx version was right this time.

That transaction is invalid because it is a double spend of the redeem transaction you already created.


Enhancing magicblue, moving to pygatt from bluepy?

Hi there,
First of all, great work with with magicblue! I was trying to integrate it with HomeAssistant, per your request, but ran into a few problems.
Wanting to know more about this bulb, I started digging a bit more, such as looking at traffic dumps: found on the research-branch and this article.
After learning more about BTLE and playing around with bluepy and pygatt, I am now able to do things, such as read the state of the lamp (color, brightness, on/off, etc.) Also, I think I am able to remove the need to specify the bulb-version (although I am not sure, I only have V10 bulbs.)
The downside is, though, that I was not able to read the state of the bulb using bluepy. I only got it to work with pygatt. The mechanism works by sending a command and then receiving a notification with data. While pygatt nicely handles the notification by calling a provided callback, I see no callbacks being called in bluepy.
I would like to improve magicblue, and using pygatt instead of bluepy would be the first move. Are you willing to merge a pull request which does this? The PR will include other finding/enhancements as well, of course. I will try to keep the existing API in tact/compatible.

Hello Steven,
The bulb version is required because versions >= 9 use a public address type whereas versions < 9 use a random address type. So except if we bruteforce the connection, we won't be able to connect to read bulb's version without specifying it.
BluePy has caused me some trouble in the past, with breaking updates and difficulties to install on a lot of platforms. If PyGatt provides the same functionalities and allow us to implement more, I would definitely be willing to merge such pull requests.
One thing that worries me though, PyGatt specifies it runs on Python 2.7. If it's not compatible with python 3, that would be a breaking change. Also I think that HomeAssistant runs on python3.

Hello Benjamin,
Thank you for your response. You are right about the public/random address issue. It slipped my mind, but I will see what I can do about it.
I cannot guarantee that PyGatt will not give issues though, like BluePy might give. The issues for Bluepy probably arise from compiling the helper binary. I think this is something we will have to find out. You can try to install PyGatt yourself and see if you have any problems during installation.
PyGatt also runs on Python 3, which I'm currently doing. HomeAssistant itself runs on Python3, indeed.
I hope to come up with something soon.

For BluePy it indeed comes from the compilation of the helper binary. From what I've tested it needs to be installed as root to compile this helper properly. If you run pip install bluepy without sudo (for example to install it in a virtual env) it will not be compiled, resulting in the error #11.
Glad to hear PyGatt also works with Python 3 :ok_hand:

Closing the defect. Please use the pull request for comments.


Fixed positioning bug when scrollbar present

@AuspeXeu Thank you, it was a terrible bug!

I see the reason why you state that. Firstly - The latest code in github has clearly a bug and there is some configuration issue from your side where bower always gives the latest version from github even if you ask for older version. It always gives buggy 2.3.8, even you ask for any older version like 2.3.5.
The main issue is setting the trying to set a value to undefined value of top.
I am looking at your demo site it use JS (http://www.malot.fr/bootstrap-datetimepicker/bootstrap-datetimepicker/js/bootstrap-datetimepicker.js) which has working condition code:
if (this.pickerPosition == 'top-left' || this.pickerPosition == 'top-right') {
top = offset.top - this.picker.outerHeight();
} else {
top = offset.top + this.height;
}
But once you download from bower or have a look at latest code in grunt (https://github.com/smalot/bootstrap-datetimepicker/blob/master/js/bootstrap-datetimepicker.js), it has below code
if (this.component) {
top = top - containerOffset.top + 169;
left = left - containerOffset.left + 210;
} else {
if (this.pickerPosition == 'top-left' || this.pickerPosition == 'top-right') {
top = offset.top - this.picker.outerHeight();
} else {
top = offset.top + this.height;
}
}

I see the reason why you state that. Strangely the version you are using in demo website and version from bower are quite different. Bower version 2.3.5 seems to be stable where as 2.3.7 and 2.3.8 are broken.
Demo site used JS (http://www.malot.fr/bootstrap-datetimepicker/bootstrap-datetimepicker/js/bootstrap-datetimepicker.js)  has working condition code:
 if (this.pickerPosition == 'top-left' || this.pickerPosition == 'top-right') { top = offset.top - this.picker.outerHeight(); } else { top = offset.top + this.height; }
But once you download from bower 2.3.8 or have a look at latest code in grunt (https://github.com/smalot/bootstrap-datetimepicker/blob/master/js/bootstrap-datetimepicker.js), it has below code
top is undefined before this code runs, it gives out NaN.
`if (this.component) { top = top - containerOffset.top + 169; left = left - containerOffset.left + 210; } else { if (this.pickerPosition == 'top-left' || this.pickerPosition == 'top-right') { top = offset.top - this.picker.outerHeight(); } else { top = offset.top + this.height; }


Update demo gif with delays and clear screen after logical step

Signed-off-by: Dinesh<EMAIL_ADDRESS>Fixes #3

Note: Size of the gif is 30MB. We should see how quickly it loads in homepage. It loaded pretty fast for me. Please test it once before merging this.

Instead of gif, can we go with video https://vimeo.com/471676863 ? so that users can pause and see the details

We could also do a overview video with audio. I saw something like that in sysdig repo which was very nice.
sysdig video link.

I added more delays and added some text which will provide context to the users. Now gif will provide good overview.
Please see the gif and provide your inputs.

@arunvelsriram we can create a youtube channel and do a voice over video for different use cases


Tidy up testcases.js

Numerous mostly-reformatting changes to server/tests/testcases.js:

Run clang-format on the file to fix various formatting sins.
Remove extraneous blank lines between testcases.
Reduce simple testcases to a single line (and put source on the same line as the src: tag).
Remove names from testcases where it is the same as src or where it does not contribute any useful information about the purpose of the test.
Fix formatting of the testcase source code itself:

Consistently indent source code by +2 spaces compared to surrounding testcase.
Apply certain styleguide rules regarding whitespace, line breaks, etc.

Make selective improvements to testcase naming and comments.
Don't use table-driven tests (within a single testcase) unnecessarily.

There are still some cases of table-driven tests where we are checking for certain errors being thrown, which is harder to to concisely as a top-level testcase.

Also:

Backport a doc fix for regexps.identifier in server/code.js
Add two tests concerning setting anonymous FunctionExpression .name properties by property assignment.

OK, it took a few tries to get everything committed the way I intended, but this is now ready for review.


[Bug]: Installing dashboard plugin repository-s3 -> AccessDenied

Describe the bug
I'm trying to install plugin repository-s3 with command bin/opensearch-dashboards-plugin install repository-s3 but says
v16.20.0
Attempting to transfer from repository-s3
Attempting to transfer from https://ci.opensearch.org/ci/dbc/distribution-build-opensearch-dashboards/2.8.0/latest/linux/x64/tar/builds/opensearch-dashboards/plugins/repository-s3-2.8.0.zip
Plugin installation was unsuccessful due to error "No valid url specified."

If i try the url with curl:
curl -fsSLI https://ci.opensearch.org/ci/dbc/distribution-build-opensearch-dashboards/2.8.0/latest/linux/x64/tar/builds/opensearch-dashboards/plugins/repository-s3-2.8.0.zip
HTTP/2 302
content-length: 0
server: CloudFront
date: Tue, 18 Jul 2023 20:29:33 GMT
location: /ci/dbc/distribution-build-opensearch-dashboards/2.8.0/6182/linux/x64/tar/builds/opensearch-dashboards/plugins/repository-s3-2.8.0.zip
cache-control: max-age=3600
x-cache: Hit from cloudfront
via: 1.1 fe41458437820e017e1a964e74df0578.cloudfront.net (CloudFront)
x-amz-cf-pop: MAD56-P2
x-amz-cf-id: MzqYueet5gZwjk8_zCu7HmKKW1V2YJ4RVNgZV90_Uil7UdGVYd3ayw==
age: 60

HTTP/2 403
content-type: application/xml
date: Tue, 18 Jul 2023 20:30:33 GMT
server: AmazonS3
x-cache: Error from cloudfront
via: 1.1 fe41458437820e017e1a964e74df0578.cloudfront.net (CloudFront)
x-amz-cf-pop: MAD56-P2
x-amz-cf-id: g3lM2tGqt6alIbBKSJNrlII99Nm3eJ3IH06zUIVnfx4aD50P2x6Hsg==

curl: (22) The requested URL returned error: 403

And if i user a browser the result is
<Error>
AccessDenied
<Message>Access Denied</Message>
<RequestId>CD6WGVT0V32S9B62</RequestId>
<HostId>
5MRfScVgm7YbJTXyKOjpfFQ7EJrXsbLBqkT1FOkj1VYTz6J/5rLbpX67pTwrOXrEnERAO10vrQ0=
</HostId>
</Error>

To reproduce
bin/opensearch-dashboards-plugin install repository-s3
Expected behavior
To be repository-s3 installed in opensearch dashboard
Screenshots
If applicable, add screenshots to help explain your problem.
Host / Environment
kubernetes with opster opensearch operator
Additional context
No response
Relevant log output
No response

Hey @raulpuente this is an OpenSearch plugin please try with  opensearch-plugin cli.
bin/opensearch-plugin install repository-s3
This was tested for 2.8.0 release.
https://github.com/opensearch-project/opensearch-build/issues/3434#issuecomment-1579292048
@bbarani @peterzhuamazon
https://opensearch.org/docs/latest/install-and-configure/plugins/

Please use bin/opensearch-plugin install repository-s3 within OpenSearch distribution to install this plugin.

Thank you but there is not this binary on the image docker.io/opensearchproject/opensearch-dashboards:2.8.0 , i only find  "opensearch-dashboards-plugin"
$ ls -lsa
total 28
4 drwxr-xr-x 2 opensearch-dashboards opensearch-dashboards 4096 Jun  3 03:10 .
8 drwx------ 1 opensearch-dashboards opensearch-dashboards 4096 Jul 23 18:28 ..
4 -rwxr-xr-x 1 opensearch-dashboards opensearch-dashboards  902 Jun  3 03:10 opensearch-dashboards
4 -rwxr-xr-x 1 opensearch-dashboards opensearch-dashboards  840 Jun  3 03:10 opensearch-dashboards-keystore
4 -rwxr-xr-x 1 opensearch-dashboards opensearch-dashboards  880 Jun  3 03:10 opensearch-dashboards-plugin
4 -rwxr-xr-x 1 opensearch-dashboards opensearch-dashboards 3491 Jun  3 03:10 use_node

The repository-s3 is an OpenSearch plugin (not OpenSearch Dashboards), so you should be able to find the plugin inside the OpenSearch docker image

Thank you, you are right. Know it works.

Thanks for the update @raulpuente, closing this issue, please re-open if required