Feature/no iraf checkpsf

This implements a interactive PSF viewer in matplotlib, removing the iraf dependency from the checkpsf step

@griffin-h check the performance for both filled mesh and unfilled mesh surfaces.

I think most of make_psf_plot can be replaces with the data extension of the PSF file in the following way:
psf_model = fits.getdata(psf_filename)
Y = np.arange(psf_model.shape[0])
X = np.arange(psf_model.shape[1])
X, Y = np.meshgrid(X, Y)
surf = ax.plot_surface(X, Y, psf_model, antialiased=True, lw=0.25, color='k', ec='w')

This puts the origin at the edge, but it should be fairly trivial to adjust that as needed (also, I'm not sure X and Y are consistent or that it matters)

I think most of make_psf_plot can be replaced with the data extension of the PSF file in the following way:
psf_model = fits.getdata(psf_filename)
Y = np.arange(psf_model.shape[0])
X = np.arange(psf_model.shape[1])
X, Y = np.meshgrid(X, Y)
surf = ax.plot_surface(X, Y, psf_model, antialiased=True, lw=0.25, color='k', ec='w')

This puts the origin at the edge, but it should be fairly trivial to adjust that as needed (also, I'm not sure X and Y are consistent or that it matters)

Is this true @abostroem? I thought the data extension contained only the residuals.

I'm totally wrong about the PSF file - it isn't that staightforward - please ignore the comment

@griffin-h check the performance for both filled mesh and unfilled mesh surfaces.

I did not find that plot_wireframe was any faster than the current version, and I agree it was harder to understand.
I realized though that the limiting factor for the speed was displaying the images in ds9, so I switched the image display to Python using some code I had previously written for a different stage. If someone wants to review those changes (@jfrostburke ?), I think it's ready to merge any time.

Tested and the iraf-less image display looks good as well. This will definitely become the default way I use -s checkpsf.


[ New Button ] : Glass Button [GSSOC'23]

This is a(Cool Glassy Button):

[ ] Update to an existing Animated Button
[ ] Any Error
[x] Proposal to the Repository
[ ] Documentation / README.md changes

**Details:
"Introducing the Glassy Button!" Elevate your Website/Project experience with this attractive and innovative button that allows you to effortlessly catch the attention of the users. **

@PoorviShetty @Spyware007 could you look in this matter and assign me this issue.

@PoorviShetty @Spyware007 Could you plz assing me this issue???


validating component tag name

closes https://github.com/donejs/generator-donejs/issues/143.

Coverage increased (+0.02%) to 97.742% when pulling 0308ae9305640b32cc5b9ae0c7ee55fc295c1ffb on name-validation into ae46b87970a659592d8c6abb83b28f91d22b5c22 on major.


Duplicate email sent on Xiaomi Mi Mix2

When sending emails through K-9 on a Xiaomi Mi Mix2, occasionally the same email is sent twice. This happens both on Wi-Fi and mobile data connections. It is unclear what triggers the issue, but it appears specific to K-9 running on a Xiaomi Mi Mix2, so either the application or the phone, or the combination of the two, may be a cause. The problem does not occur when sending emails through Xiaomi's native email client using the same account, or when using K-9 with the same account on a Xiaomi Mi4 and Xiaomi Mi Max2.
Steps to reproduce

Send email from within K-9.
Check "Sent" folder of account where the message is sent from.
Check inbox of recipient account.

Expected behaviour
One email is sent. The "Sent" folder should contain one copy of the message. The recipient should have received one copy of the message.
Actual behaviour
When the issue occurs, two emails are sent. The "Sent" folder shows two messages with identical subject and body text but with different IDs and different time stamps. The recipient has received the same message twice.
Environment
K-9 Mail version:
K-9 Mail Version 5.600
Installed via F-Droid
Note: K-9 is given full permissions and battery management is set to "unrestricted".
Android version:
Xiaomi Mi Mix2 (Chinese version)
MIUI <IP_ADDRESS>
Android 8.0.0
Security patch level 2018-12-01
Kernel version 4.4.78-perf-g7ac6a25
Account type:
IMAP account at FastMail
Log
A log illustrating the issue is attached. The problem occurs at timestamp "03-02 11:23" (two emails are sent), but it does not occur at "03-02 11:34" (only one email is sent).
k9-log-subset.txt

They are two different messages. Maybe two clicks are registered and two emails are generated.

An interesting thought! If there is anything I could try or do to confirm if that is the case, please let me know.

When sending mail, K9 always creates a double sended message in the sended mailbox. This happens with a outlook account, I'm using imap in K9
Samsung Galaxy S9 Android 9

This happens also depending on server account.
For example, outlook/hotmail will also produce duplicates sent messages because outlook/hotmail server changes message ids for sent messages.
https://github.com/k9mail/k-9/issues/2736

This is not clear: https://github.com/k9mail/k-9/pull/3598#issue-213032549
So, for outlook accounts we need to remove sent folders?

I came across these bug reports as well, but I think my issue may be different, since it only occurs on my Xiaomi Mi Mix 2, but not on any other devices.
I noticed that I tend to press the Send button twice in quick succession, a subconscious habit acquired from overriding the empty subject warning ('Subject is empty, click again to send anyway'), but I do it even when it's not actually necessary. Experimenting with this just now it seems that this is indeed what triggers the duplicate email to be sent. Why this only happens on the Mi Mix 2 is unclear though.

I've had this issue occur to be both ways (as the sender and recipient), with a Samsung Note 8, and Samsung Galaxy S9, on either side.
I think the problem is caused by tapping the send button twice. The reason this is possible in the first place is because when tapping the send button, nothing happens, so the user is tempted to tap it again and that causes the email to be sent twice. A solution would be to disable the send button once it's been tapped once.


lexer.ts: added lookup, fixed indentation problems

whoops, it's not working, one moment i need to find out what's wrong

Alright I fixed it; I accidentally put a break in the wrong spot making it instantly exit the parse loop (and I forgot a break after the case "=").

I will merge this in the next version (when I have time)

Alright, I also added negative and decimal numbers for it, since it was only a few changes. (Unless you wanted it by design to not have those?)


Prometheus metrics library upgrade

PR Description
This PR upgrades the prometheus followed by the upgrade done by besu (on main branch).
It replaces the deprecated method MetricSystem::createLabelledGauge by MetricSystem::createLabelledSuppliedGauge. Uses the definition of histogram created in the besu plugin and fixes tests. Since this introduces breaking changes and it depends on a release of the new metrics version of the besu plugin we might want to hold this PR until the next besu release comes out.
Fixed Issue(s)

Fixes https://github.com/Consensys/teku-internal/issues/168
Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

[x] I thought about adding a changelog entry, and added one if I deemed necessary.

Docs PR https://github.com/Consensys/doc.teku/pull/624


8: Expand fgs system tests

Fixes #8
Note, is branched from https://github.com/DiamondLightSource/python-artemis/pull/7 so merge that PR first.
To test:

Ensure you have https://gitlab.diamond.ac.uk/controls/python3/s03_utils/-/merge_requests/4 running for the fast grid scan (this is not yet on ws019
Run the system tests and confirm they all pass

Thanks for the review, changes made. I also found that despite the new changes being on s03 the tests still fail. I would normally look into fixing it but i03 testing takes priority so have fudged it and created https://github.com/DiamondLightSource/python-artemis/issues/13 to fix properly


Add compile option 'JDBC_EXTENSIONS'

If SQLite has been augmented with JDBC extensions add a compile option
which can be queried from the 'compile_options' pragma.
This makes it possible to determine at runtime whether the client supports
customs functions or not.

@xerial Done, please have a look.

ok. Merged. Thanks!

@xerial Brilliant, thank you!


Support override_stream_error_on_invalid_http_message

Description:
Envoy Docs:

if you run level two Envoy version 1.11.1 or greater which terminates HTTP/2 or above, we strongly advise you to change the HttpConnectionManager configuration of your level two Envoy, by setting its downstream validation of HTTP messaging option to true

Envoy Gateway should support configuration of this setting, so that users can adjust it, based on their deployment mode (L1 or L2) and protocol versions in use.
Relevant Links:
https://www.envoyproxy.io/docs/envoy/latest/configuration/best_practices/level_two

closed in favor of #3682


World freeze

I updated to the latest 1.4 version in a heavy world gen modpack I've been working on and it slows world gen and eventually freezes it as well as the mobs. I can still interact at times. It doesn't crash. I can log out and eventually it closes (takes some time). Again, with no crashes. I reverted all changes and this mod is the only one that seemed to be causing the issue. shrug I have no idea. EDIT: 1.4 version

Cancel that. I've traced it back to Maiden's Marvelous Materials. Sorry for the panic. I disabled the file but it somehow stayed enabled during my testing. Apologies.


Compute CSA on GT and output prediction mask

Context
We want to compute CSA on the GT (hard and soft labels) in order to get the initial variability and also on the output prediction mask.
In order to do so, we need also need the discs labels to compute CSA at the same levels on all contrast. Therefore, vertebral levels need to be identified on an anatomical image (T1w or T2w) and propagated on the other contrasts. Since we have the warping fields of all contrast to T2w, it is possible.
Proposed method:
For CSA on GT, in process_data.sh

Add vertebral labeling on T2w if label doesn't exsits
For all contrast perform resgistration (remove current even if softseg exists)
Bring disc labels to each contrast space
Crop the disc label
Copy cropped label into derivatives of cleaned dataset

For CSA on prediction mask (create a new processing script to compute CSA):

Use disc labels available in the derivatives to compute CSA on the output prediction mask.

@sandrinebedard any updates on this?
Currently, in the plots I am generating, I am using the .csv files found in the results folder of the data_processed_clean folder. BUT, the issue is that these csv files contain the CSAs for all the subjects and we're plotting it along with the model prediction CSAs which only contain subjects from the test set. Basically, it is not an apples-to-apples comparison.
Do you have any suggestion on how we can generate the GT csv data only for the subjects in the test set?

Maybe we could load all CSV files (the ones with all the subjects and the ones containing only subjects from the test set) as Pandas DataFrames, and we could use test subjects to filter out subjects from all the subjects.

I would go with @valosekj's suggestion, to do the filetring inside the script

Seems like this has also been completed since we now have plots with CSA on the GT segmentations. @sandrinebedard could you please link to script or part of the code where this is computed now? Thanks!

Is done in https://github.com/sct-pipeline/contrast-agnostic-softseg-spinalcord/blob/e4171e8ce39bafbb592f1263481107071f8bf3e8/csa_generate_figures/analyse_csa_all_models.py#L295-L297


Storage Backend: Google Drive

This is a feature request for Google Drive as a storage backend.

Thanks for the suggestion.

FYI
https://developers.google.com/drive/v3/web/quickstart/go
https://github.com/google/google-api-go-client/blob/master/drive/v3/drive-gen.go

any short term plans to implement this?

To be honest with you, there are no short-term plans for this. Sorry.

The people from the rclone tool have created an drive backend here: https://github.com/ncw/rclone/tree/master/drive

Hi, any possible news on this?
The latest Crashplan changes have made me look for an alternative and the unlimited space available when using G Suite for Business is really interesting - and restic seems like a better alternative than Arq or Duplicati since I can run it natively on my FreeBSD box..

There are no news here, it's one of the more problematic backends, requiring a client secret (that should be kept secret) which needs to be embedded in the Go source code.
You can make a local backup and move that to Google Drive with e.g. rclone.

@fd0, from https://developers.google.com/identity/protocols/OAuth2InstalledApp , it seems to me that if this  mode is used the client secret do not need to be kept confidential.

@fd0 Alas, that's my initial problem - with 6TB+ of data that is not really an option..
As @tamalsaha mentions others are using OAuth without problems.
The team behind Duplicati has even published this explanation: https://www.duplicati.com/articles/OAuth/ and uses https://duplicati-oauth-handler.appspot.com/ to request/revoke tokens.
The explanation is based on OneDrive, but it's the same procedure with Google Drive and their OAuth-handler lists a lot of providers based on OAuth.

Hi guys, I started to play around some time ago with the restic source code to give it a shot to integrate gdrive API package. But there is a lot of duplication going on (needed to change 3 places to add a backend). I have not proceeded because I moved on to some other stuff. https://github.com/xor-gate/restic/compare/master...add-gdrive

With Google File Stream you might be able to back up restic today using just the GFS mount point?

This backend can be used via rclone, I'm therefore closing this issue for now. Please feel free to add further comments. Thanks!


Expose various color settings and precession chart

This PR exposes the color settings added in https://github.com/WorldWideTelescope/wwt-webgl-engine/pull/258 to pywwt, as well as exposing the ability to toggle the precession chart. This required bumping the research app to the newest version. I've added two new steps to the full widget test to check that these settings are working correctly.
As for the naming, I decided to go with <gridtype>_grid_color (e.g. alt_az_grid_color) rather than <gridtype>_color because this would be ambiguous in the case of the ecliptic vs. the ecliptic grid.

I've fixed the copy/paste-o and some small codestyle issues. Not sure what happened with the image test - I must've missed committing one of the CI image files before, since it's fine after adding the last Windows output image.


Optimise binary string parsing by rewriting it to binary manipulation.

Coverage decreased (-0.2%) to 83.626% when pulling 1f6abfc924afa3cdedf5008c13846b443b1426f4 on kpy3:optimize_binary_to_decimal into cf7b72374657b6c414585e725c310427f6897953 on egobrain:master.

Coverage increased (+0.4%) to 84.211% when pulling 500d780de5c7da813ba9b3ba61f9a974257f497b on kpy3:optimize_binary_to_decimal into cf7b72374657b6c414585e725c310427f6897953 on egobrain:master.


Navigate to flutter screen

Ticket: #440
EDL remains the same i.e pass the screen name.
Only configuration needs to be done on flutter side that is pass custom widgets along with name.
Example:
void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  // (optional) pre-initialize asynchronously so Ensemble is ready
  // when your app switches to Ensemble screens. Add `await` before
  // this statement if your first page is an Ensemble's page.
  Ensemble().initialize();

  /// (optional) set map of custom flutter widget that can be called from ensemble
  Ensemble().setCustomFlutterWidgets(
    {
      'testScreen': (context, args) =>
          TestScreen(name: args?['name'] ?? 'Hello'),
    },
  );
  runApp(const MyApp());
}

EDL to navigate to TestScreen will be no different,
- Button:
    label: Test Screen
    onTap:
      navigateModalScreen:
        name: testScreen
        inputs:
          name: PageName

Thanks @snehmehta how is the opposite interaction achieved i.e. from Flutter to a particular Ensemble screen? Let's get that done as well if it is not a major effort

@snehmehta the only issue I see is if there is a name conflict between a screen in my app vs the custom widget I am passing in. Secondly, how would the preview distinguish between 'testScreen' missing or 'testScreen' being a custom widget as the preview doesn't know about flutter custom widgets. It should degrade gracefully.
Suggest adding a parm such as -
- Button:
    label: Test Screen
    onTap:
      navigateModalScreen:
        name: testScreen
        type: externalWidget
        inputs:
          name: PageName

I am intentionally calling it externalWidget so as not to tie to Flutter only

why is this PR still open? I thought this was completed a while ago? @snehmehta ?

@snehmehta please close this if it is done


Refactoring 1_stdlib/Optional.swift test file

Can somebody have a look at the PR and leave some comments and advices about code style, variable naming etc.
I use ./llvm/utils/lit/lit.py -sv ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/1_stdlib/Optional.swift for running tests. Is it right way to do so?

@gribozavr thanks for review. Can you look again :)

Whats about Optional<Int> vs Int? What do you prefer?

Int?, where parser accepts it.

@gribozavr Optional<Int> and Int? are they equal?

@frootloops Yes, T? is syntax sugar for Optional<T>.

@gribozavr ok, thanks.

@gribozavr is PR ready for merge?

@gribozavr I have update the PR.

@frootloops Thank you!


Improve readability of documentation

@chhoumann
I am very glad my contribution was accepted and was not in vain! :-)
Since you welcomed my contribution, I have a further suggestion that I didn't dare to implement since I didn't know how well my PR was to be received. It concerns the Multi Choice. Using the "choice" in the name is highly confusing because it really has nothing to do with the other 3 choices.
Can I get your initial approval to brainstorm alternatives, change that particular name, and make other changes that would make the documentation clearer?
You don't have to promise to merge my changes automatically - obviously, you will have the opportunity to see and decide when it's time. But can I get your initial approval to change that particular name to something more fitting?

@mikkovedru I'd be more than happy to hear your thoughts on alternatives. :)


Extend Datatable row which have the ability to show/hide other component(Card or something)on click

**I'm submitting a
[ x] bug report
[x ] feature request

**Current behavior**
Hi Team,

At this moment for the exisiting datatable with records, i am not able to extend the datatable row.
Which means on click of datatable row, the row should expand and display more information about that row.

Is it possible to extend the row clicked like below.

![image](https://user-images.githubusercontent.com/17860925/28642764-1e4dcf6e-7219-11e7-8e4d-5445ac648a11.png)

Table Version : 9.3.1
Angular : 4.0.0

Any update on this here please?


Cannot edit requirements

This needs to be confirmed, but I was messing around with our Heroku demo and don't think I could edit the requirements a user has fulfilled. My guess is that this is a simple strong params issue, but it needs to be replicated first:

Log in as admin/superuser
Visit user profile (yours or someone else's)
Attempt to add requirements (select them in the list)
Click "Save"
Upon clicking edit again, no requirements are selected

I didn't exhaustively test this, but we should figure it out and fix it if it's an issue. Sticking this in v5.5.0 since it would be a pretty sizeable bug.

This has been verified locally on master, not sure what's going on. I'll spend a few minutes debugging but if I can't figure it out we'll have to deal with this ASAP.

It does appear to be a strong params issue but I can't figure out why it's not including requirement_ids as permitted. We're restricting it to those who can? :assign, :requirements, but superusers and admins should both be able to do that...

Ok, figured it out; it's an array thing. One issue - it seems as though checkout persons and up wouldn't be able to create a reservation for a user for restricted equipment for which that user has met the requirements since we disable all the add to cart buttons if the current user doesn't meet the requirement, not the reserver. I'll update that in this issue as well.

All done, with spec. Issuing PR now.


Toggleable Metrics/Labels

When running Renovate against a large number of repositories, it can lead to rather high cardinality in the metrics that are being produced by renovate-metrics. This is due to the many labels with a potentially infinite set of values. It would be nice if there was a way to toggle certain metrics/labels off to help reduce the number of metrics.

I don't think it make sense to support this. If you really want to ignore certain metrics you should drop them at scrape time by using metrics relabeling.


Add Try Operator support

This will address #692, but I have some questions about how best to proceed.

This is passing on stable and nightly on my box right now; not sure why the tests are failing on Travis yet.


maxLength propType requires string,

Hello
Loving react-tags :)
While implementing the library i noticed that maxLength requires a number in the docs but in ReactTags.js a string is required.
PS: Created a PR.

PR Accepted thanks :)
#140

PR Accepted THANKS :)
#380


Explore adding an event for the staking pallet's chill_other

The chill_other extrinsic does not emit any events. Adding an event may help improve the ergonomics of researching the staking apparatus. In particular it would make it easier to find instances of the call in explorers that support event based lookup. Many explorers do not support lookup of nested calls (i.e. those made within batch/proxy etc).

@emostov yeah just add one for all the chill events.


Autofocus of camera in AR

I started this question on the Babylon Forum (https://forum.babylonjs.com/t/babylonreactnative-how-to-get-ar-camera-to-focus/20588)
Is your feature request related to a problem? Please describe.
Currently, on Android autofocus is automatically disabled (https://developers.google.com/ar/reference/java/com/google/ar/core/Config.FocusMode). This makes it so that looking at small objects close-by, the background is blurry and this takes away from the experience. I find myself waiting for it to focus.
Describe the solution you'd like
It would be great if there was the option to enable autofocus in the camera in Babylon. So you could set it when creating the camera.
If this is not possible, having it consistent with ARKit which (for iOS 11.3 and later) has this enabled by default (https://developer.apple.com/documentation/arkit/arworldtrackingconfiguration/2942262-isautofocusenabled) would also save some confusion.
Additional context
I couldn't find any guidance on this in the OpenXR specification, neither did I find anything in the WebXR specification. This issue does come up with WebXR, but there's not been an answer to it (https://github.com/immersive-web/webxr/issues/1154).
Usecases that would benefit from having autofocus are all usecases that need to have people viewing things close-by. For example, an ad with 3D objects on top of a magazine, viewing a smaller AR interactive model of industrial machinery on top of a table or any architectural scale model.

C++ isn't something I have experience with. But I did find what the config in ARCore should look like, to give an idea.
            ArConfig *config;
            ArConfig_create(session, &config);
            ArConfig_setFocusMode(session, config, AR_FOCUS_MODE_AUTO);
            ArSession_configure(session, config);

Inserted at line 450 of Dependencies/xr/Source.ARCore.XR.cpp in the Initialize() function makes it autofocus. The config should then probably be destroyed when the session is.

This has been fixed (for now hard coded to have auto focus on - in the future we might plumb through an API so JS can control this).


EngineSpec Refactor

The gist of and motivation for this change is summarized in this notion document
The general idea is to replace the Specs Engine enum with the following:
type Engine struct {
        // Type is a string representation of the Engines type
	Type string
        // Params is the set of arguments(?) the engine will use during operation.
	Params map[string]interface{}
}

Using a plain string for Type instead of dedicated enum type (as we do now) allows for the addition of new engines and submitting jobs at runtime without having to do a new release or approve the engine. For example, a compute provider wants to accept spark_3.3 jobs. It should be able to do so just by publishing the engine in it's NodeInfo message. Users should be able to submit spark_3.3 jobs without having to update their client to wait for the requester to update their server.
Using a map[string]interface{} for Params instead of a structure containing dedicated types (as we do now, e.g. EngineSpecDocker) enables the addition of new engines as their arguments will be dynamic to the engine Type. In future version we may consider making this a []byte, but for now a map is more pragmatic.
Acceptance Criteria

[ ] Changes to EngineSpecDocker

[ ] move from the modules package to the executor package.
[ ] move the EnvironmentVariables field to the Spec structure.
[ ] provide conversion between the Engine.Param and concrete EngineSpecDocker type

[ ] Changes to EngineSpecWasm

[ ] move from the modules package to the executor package.
[ ] move the EnvironmentVariables field to the Spec structure.
[ ] rename EntryPoint to Entrypoint
[ ]  provide conversion between the Engine.Param and concrete EngineSpecWasm type

[ ] Delete JobSpecLanguage.

[ ] Delete its executor and related bits

In addition, we must determine whether we should permit the creation of "invalid jobs" through the Command Line Interface (CLI). At present, during job creation, we validate its engine against a list of supported engines. In principle, this change would eliminate the notion of "valid jobs," as it grants users the flexibility to specify any engine name of their choice. Instead of rejecting jobs with unrecognized engines, we could issue a warning to users about the utilization of an "unknown" engine.
For now I think it will be easiest to leave this validation in place, and remove it in a follow on when we are ready to make the network more open.

Definitely support the idea! A few questions:

Do we have plans about how to describe the schema of executor specs for use by the CLI? Atm one of our core design choices is having a CLI that is accessible to not-particularly-technical users. E.g. bacalhau docker run mirrors docker run very closely, and that is very executor specific, and I don't think we want to lose the ability to provide a very specific UX. Are we just going to leave the CLI as is for now and revisit this later?
What is the motivation behind moving EnvironmentVariables into the generic Spec? It's not guaranteed that all executors have a meaningful way of using them.

The fact that they are common between Docker and Wasm atm is more an artifact of our Wasm support being POSIX-like, but we could definitely support Wasm runtimes that don't have a concept of env vars in the future.
My acid test for "is it too specific to our current executors" is imagining an excel-2016 executor that can do something meaningful with spreadsheets (like run calculations) – env vars don't neccessarily make sense in this context.
So can we just leave them in the individual executor specs?

What are we going to do about Spec.AllStorageSpecs()? It is meant to return pointers to all the storage objects contained within the Spec, to allow them to be generically transformed. E.g. right now if a WASM blob or Docker container is submitted inline, it can be saved to IPFS if it is too large. We also have open issues about doing the same with URLs, e.g. #1888. I don't think we want to go back to having a big bucket of storage specs, because we specifically refactored away from that due to it being difficult to use.
~Do we know how are we going to handle executor specific transformations by the requester?~ Such as #2255? I guess we can just have executor-specific code there as we would have to currently.

Are we just going to leave the CLI as is for now and revisit this later?

Yes that seems easiest at the moment. My thinking is we'll translate the job arguments from the CLI as they exist now into the Params map[string]interface{} field. The keys of the map will correspond to the field names in the specific job spec type e.g. EngineSpecDocker. Compute nodes supporting the engine type are then free to transform the Params map to a concrete type for execution.
The gist of the current change will be to store the CLI args in command specific variables rather than decode them directly into a job spec as we do now.

What is the motivation behind moving EnvironmentVariables into the generic Spec?

Motivation was initially stated here. However, after giving the doc a closer look I remembered we both disagreed with the change to move them to Spec.

My acid test for "is it too specific to our current executors" is imagining an excel-2016 executor that can do something meaningful with spreadsheets (like run calculations) – env vars don't neccessarily make sense in this context.

In that specific case, assuming we did move EnvironmentVariables to the Spec, they would just be null. However, that would mean Requester nodes need to supply compute nodes with the Spec in addition to the EngineSpec, which we do now, but may not be what we want in the future.
(An aside: I worry users may not realize Job's (and their bits) are communicated over the network in plain text and that putting seneitive information in the EnvironmentVariables means leaking it to the world.)
@wdbaruni what do you think?

What are we going to do about Spec.AllStorageSpecs()?

Since the method is utilized by Requester nodes, it causes the contents of an EngineSpec to be exposed/leaky as the Requester node needs to inspect it. At present, this is acceptable due to the limited and well-defined set of Engine types. However, once custom engines become supported, the Requester will not have the ability to inspect the contents.
Since the JobSpecLanguage engine is being removed, the JobSpecWasm is the only one needing inspection (the JobSpecDocker isn't inspected using this method). It wouldn't be hard to preserve the methods functionality for the time being. But, in general, having a Requester node modify job specifications before sending them to compute nodes is not ideal, and probably won't be scalable as the network expands.
I think we could either:

reject jobs with inline data larger than 5kb (our current limit), which I think means limit the total size of the EngineSpec blob.
perform the transformation on the client side since the client ought to be aware of the EngineSpec they are issuing.

Either of the above would allow this method to be removed, and reduce the responsibilities of the Requester node.

Do we know how are we going to handle executor specific transformations by the requester?

In the same vein as the above point, I think we should move away from request nodes modify the job spec (with in reason) before passing to the compute nodes.

So I agree with the top two responses – thanks.
However, we seem to be adopting this idea that the requester node is both completely executor agnostic and also does not/can not modify the user's submitted spec, and I think this is worthy of more inspection.
We have a number of features today/planned that involve the requester modifying the spec in an executor-specific way. All of these features are supporting an easier UX, better network reliability, or reproducability. For example:

Examples of features we have/need
• #1628 – allowing the user to submit (large) storage blobs inline from their local environment which are then converted into (small) IPFS references. The use case is for thin clients (such as web browsers) to be able to include up to 10Mb of data with their request which is easier for them than spinning up a full IPFS node or pinning it to some external storage first. This is also intended to cover things like scripts or code that the user is developing locally but may be changing job-to-job.
This is definitely used today for WASM blobs (at least 2-3Mb in size) and for sending Python code from Jupyter notebooks (often more than 5Kb), and is of interest for other edge applications (e.g. a webcam node can submit an image along with the job to process the image). This involves looking inside the spec and being able to identify storage specs.
But we don't necessarily want blobs of this size traversing the network and we almost definitely don't want to be sending out blobs like this in a gossipy way. So the requester saving these blobs off to some appropriate storage is a feature for UX and network reliability. There isn't really a way to replace this by getting the client to do something, and limiting the size of inline data to 5kb is not meeting the requirement.
• #1888 – allowing the requester to cache URL downloads for concurrent jobs. This is a reproducability requirement – otherwise the data used in the job is not really captured in the job spec (assuming the general case of non-idempodent HTTP results).
It is also a determinism requirement – the powerful principle that we should retain is that "the same job running concurrently over all nodes should be doing the same work at every node", which is the basis for determinisitic verification. Right now we allow different compute nodes to receive different URL inputs which in the general case makes it not possible to deterministically verify the results and spot bad actors. It's also a safety requirement – we shouldn't allow people to DOS remote sites by specifying some large remote input and --concurrency=1000.
So there is a job for the requester here in downloading these inputs once, caching them accessibly (e.g. on IPFS), and then ensuring each compute node receives the same CIDs. Again, in general it involves being able to identify the StorageSpecs within each spec. Again, it would be poor UX and render the URL inputs flag useless to require the client to do this just to ensure sane computation.
• #2255 – allowing the requester to resolve a Docker image into a reproducible digest. This is a bug that our real users have already hit multiple times, where they make several pushes to mytag:latest and then are baffled as to why the previous version of the image is being used. It also has the same issues discussed above for URL inputs because different compute nodes will have different cache states, so if you are running a job concurrently you may end up with nodes running the job in different images = bad. Obvious issue for reproducability also in that being able to re-run some Docker job with a latest tag is not generally possible without knowing what the state of latest was when the job was run.
We have already had some back and forth on this issue, but I've not heard any compelling arguments as to why the requester should not be able to do this, optionally. I don't think telling the users "you should use a sha256 digest of the image, actually" is practical and again to require the client to do this would require them to have Docker tools installed on their machine/web browser, which again is not practical.
• unlogged yet, but for computation involving a TEE or Secure Enclave, someone needs to verify that the signatures delivered by the Enclave are actually valid before releasing encrypted data to it. Right now, the flow being proposed by Encloud fits well in the Bacalhau architecture as another executor, but it relies on the Requester node being a trusted agent for the user (something it already is) and being able to run executor-specific code to verify sigs. If we were to push this requirement onto the user, it would massively complicate client code (essentially clients would have to run a server with a key management service) and again make these jobs inaccessbile from constrained environments like browsers and smart contracts.

Tl;dr:

We need these features and more like them to ensure network reliability, scale, security and reproducability (all things the core team should be concerned with) and the requester node is in best place to have these responsibilities through virtue of it being a user-trusted node and the one that is orechestrating the job.
Moving these responsibilities to the client either doesn't meet the requirement, would place too high a UX burden on users, or wouldn't be actionable by thin clients (web browsers, smart contracts, edge nodes).
Some of these features used executor specific functionality (i.e. resolving a Docker tag) whereas others do not (e.g. validation, moving data from one type of storage to another).

Executor-aware requesters
I completely understand that introducing generic engine types means that not all requester nodes will be able to carry out all of these functions. However, I think we would be missing a trick if we said that requester nodes cannot carry out these functions if they are able to. I agree that people should not be required to run a requester node with access to a Docker daemon, or in general requester nodes should not need to know in detail about every engine they will orchestrate, but there is clearly extra value that can be delivered if they choose to.
I don't see why we shoud not enable a higher quality of service if they want to do that. We have talked in the past about how there is a market-making opportunity in providing a higher quality of requester node service, or providing requester nodes with access to specific features like Secure Enclaves or HIPA-compliance, and we should be enabling that to happen. The analogy is with Filecoin-wrapper services like Estuary or Web3.Storage – these only exist to provide a more friendly, higher quality, managed service interface for things that people can already do with Lotus. Requester nodes or plugins to them can adopt the same role in our network.
So my proposal is that we allow requesters to run executor specific features if they want to, but do not make it a requirement. We don't have any story about plugins yet, so for now that just means that we continue to ship all of the executors in a single binary, and if the requester detects that the executor is installed it can run executor-specific transformations, else it just doesn't do that. I do not think this has a problem with scale.
Seeing inside the job spec

However, once custom engines become supported, the Requester will not have the ability to inspect the contents.

I don't think this has to be true – I don't think we have to say "executor-specific parts of the spec are opaque". After all, we have schema languages that allow data to be described in a structured manner. And, in a way that can allow inspection of it at runtime by a client that has never seen that data before.
As an example of what we could have instead of an opaque spec, we could instead specify:
type Engine struct {
    Type CID
    Spec any
}

where the Type is a CID that resolves to an IPLD schema document that describes Spec. When a node (requester or even a client!) sees a engine Type, it can download the IPLD schema from IPFS and use to perform generic transformation of the job spec. The example I'm thinking of is those storage transformations - it can just walk the tree of the schema and identify any storage fields, and then transform them generically according to its policy.
This also allows the requester to validate the users submitted Spec without needing to have support built-in – it can just use the schema it downloads to validate it. This is actually better for orchestration, because it allows the requester to reject the job rather than having to send it out to N+ compute nodes and then receive simple validation errors from all of them.
Using the CID means we also get all the familiar advantages of content-addressing, i.e. compute nodes would advertise what CIDs they support, and then there can be no confusion about naming (e.g. my python is not the same as your python) and versioning (e.g. my spec is for a v2, but your compute node is only accepting v1 – they would have different CIDs).
We also know we're going to need a story for the CLI later – an accessible schema for the engine spec could well be a key part of that support.
As ever, if the requester cannot resolve the CID, it can just skip and validation and pass the spec through to supporting compute nodes anyway.
I would welcome thoughts on that approach.

I think we should move away from request nodes modify the job spec (with in reason) before passing to the compute nodes.

Could you say a bit more about this? I've outlined a few features above where this is valuable.
Are we worried about content addressing and/or reproducability? If so, I believe that if we transform the job spec, adding a field to refer to the CID (or equiv) of v1 of a job should answer most of our questions there – it would be clear to everyone on the network that the job spec being run has been transformed, and how to find/index the original. W.r.t octostore/amplify, it would still be possible to allow the user to query using a the job spec they submitted for the executions that were carried out as part of that transformed job.

Regarding environment variables popping up to the spec level is that env variables should not be restricted to executor engines only, but we can expect storage source and publisher implementations that can accept env variables. So instead of having to define them in different places in the job spec, it might make sense to put them in one place?
However, if we believe we will move in a direction where a single job can have multiple tasks similar to IPVM invocation, then it make sense to have env variable at the task/executor level instead.
I am neutral here and don't have strong preference towards any

Going through the comments, I want to clarify that the main goal here is to allow the requester to be agnostic of the executor engines to orchestrate jobs properly, and to allow introducing new engines with minimal code change. As Simon mentioned, it doesn't mean that some Requester Node implementations won't provide added value to the by inspecting the jobs if they wish to, specially when we talk about docker which I expect to always form 90% of the jobs in the network.
In terms of job immutability and transforming the submitted job in the Requester side, today we are treating jobs as immutable after they are created in Requester Node and not necessary what the user submits. I think this is fine for now, and we can always revisit this in the future by allowing the users to pass a flag to treat the submission as immutable, which in this case the Requester will fail if the submitted job is not compatible instead of trying to make it work
However, for the Requester Node that we are implementing and maintaining, we should really think twice before allowing the Requester to inspect and transform the job, as we don't want that to become the norm, I believe it can be a premature optimization now as we don't have a lot of customers yet, and can be difficult to disable it after few are using it. Basically I prefer to focus on building a higher quality core for our product with minimal dependencies and distractions, in favour of solving every single problem that users can solve on their own with minimal effort.
The examples that were mentioned:

https://github.com/bacalhau-project/bacalhau/pull/1628 – Caching large payloads in IPFS: Based on my understanding, newly added IPFS content can take time to be advertised to the network and it will depend whether the compute nodes have direct peering with the requester to know of new content immediately. It also adding a dependency on IPFS. What if node that can execute the job (e.g. through node labeling) doesn't support IPFS? Unless we had critical use cases we wanted to support, I would've preferred users solve this problem on their side. We can explore increasing the 5kB limit to accommodate more users. We can also explore compressing the gossipsub message by replacing the large payloads with their hashes or CIDs, but without having to publish to IPFS
https://github.com/bacalhau-project/bacalhau/issues/1888 – allowing the requester to cache URL: Now we need to solve this for S3 content as well as the are also mutable. It should've been the user's responsibility to make sure the content doesn't change while the job is executing, or they can provide checksum for the content which allows failing the job if the checksum doesn't match the downloaded content, similar to what we do with S3 inputs
https://github.com/bacalhau-project/bacalhau/issues/2255 – allowing the requester to resolve a Docker image into a reproducible digest: Wouldn't querying the registry in the compute nodes and refreshing the cache if stale solve this issue without adding a dependency on the requester node? It will still be the user's responsibility to make sure the image is not updated while submitting a job with high concurrency, which doesn't sound bad to me, specially that 99% of our jobs are with no concurrency. We can also improve the user experience by sharing the digest of the image that was used to run the job after the job is completed to help them debug incidents.

In a sense, given these requirements, the Requester Nodes seems similar to an Estuary Node, but for compute rather than storage. It provides extra utility for executing jobs on Bacalhau. Sure Clients could communicate directly with Compute nodes, but doing so via a Requester node provides a better UX/Higher quality of service to the Client.

An aspect I would like to clarify more precisely is the distinction between the protocols employed in the interactions between a Requester and a Compute Node, versus those between a Requester and a Client. I anticipate that the protocol governing the relationship between a Requester and a Client will be highly adaptable, considering the diverse services a Requester can offer to a Client. Conversely, the protocol for the connection between a Requester and a Compute Node is likely to be more rigid, as it pertains to crucial factors such as determinism, reproducibility, and security. My thinking here initially fell mostly into the latter, but this conversation has made clear the value of both. (When I say protocol here I mean the structures we are passing back and forth over a transport among nodes, not the transport specifically.)

In terms of job immutability and transforming the submitted job in the Requester side, today we are treating jobs as immutable after they are created in Requester Node and not necessary what the user submits.

To this point, and my desire for more clarification above, might it be worth breaking apart protocol at the Client<->Requester layer vs Requester<->Compute layer? I think this is something worth considering, and seems like a path we are already heading down given this proposal to the EngineSpec.

the Type is a CID that resolves to an IPLD schema document that describes Spec. When a node (requester or even a client!) sees a engine Type, it can download the IPLD schema from IPFS and use to perform generic transformation of the job spec.

I am 100% on board with this design and am strongly in the camp of ""(Engine) Specifications should be self describing." And to @wdbaruni's point regarding "building a higher quality core for our product with minimal dependencies" I think this helps move us there by allowing the enhancement/support offered by the Requester to be "pluggable".

I think we should move away from request nodes modify the job spec (with in reason) before passing to the compute nodes.

Could you say a bit more about this? I've outlined a few features above where this is valuable.

I am contemplating this primarily from the perspective of protocol design. Assuming we permit Requester nodes to alter the state of a job specification submitted by a Client before forwarding it to a Compute node, I would prefer to have two distinct job specifications. One would be dedicated to communication between a Requester and a Client, while the other would be a more rigid specification designed for interactions between the Requester and the Compute node.

Some drive by comments on the github issues shared above:

https://github.com/bacalhau-project/bacalhau/pull/1628: I think this implies we need to offer some kind of storage solution along side the Requester node. As @wdbaruni states I would prefer to place this responsibility on the Client.

https://github.com/bacalhau-project/bacalhau/issues/2255: Possible unpopular opinion- we should disallow the usage of latest in the DockerEngine spec (and just in general). And this doesn't mean the Requester couldn't resolve this before running the job, but we should always be explicit about versioning, especially in the protocol, otherwise things will (probably) break or behave unpredictably.

So I think this discussion is really about the two competing priorities of making our users successful in using our product and keeping our technical complexity at a manageable level. Happily I think we are all agreed that we need to be successful in both of these domains, and the discussion is just about how we balance those competing forces in order to achieve that dual success.
My perspective is that in all of the examples mentioned, we have added/planned the feature in direct response to/observation of our users failing to use our product. In the comments above I have tried to outline why I think that passing responsibility to the client won't lead to user success. (Regretfully, saying RTFM to the users doesn't seem acceptable in 2023! 😉) What am I missing? Do you think that putting these responsibilities on the user will be successful? Perhaps we have different ideas about who the users are now?
So I would much rather have a discussion about how we can solve these problems in the least technically complex way, rather than jusy saying "it's the users problem". I'm very happy to accept the currently proposed solutions have complexity and it would be good to discuss alternative ideas that manage that downwards.
In any case, as long as we are all agreed on having a self describing spec that would be inspectable by the requester, I don't think we need to continue this discussion here, and we can kick the can down the road a little of whether or not these features are kept.
@frrist, I think your observation about there being two protocols here is a useful one. We have indeed talked about having separate API and internal job specs before, but I guess we've never done the work to actually do something separate. Very happy to accept that once the job is in the requester<->compute format, it no longer is mutable. Most of the places it is interesting to mutate jobs seem to be in dealing with user input.

In any case, as long as we are all agreed on having a self describing spec that would be inspectable by the requester, I don't think we need to continue this discussion here, and we can kick the can down the road a little of whether or not these features are kept.

Agreed on self describing spec. I will be moving forward with the (edit) changes proposed in OP.
Will IPLD-ify the EngineSpec at some point in the future as we don't need badly right now, and I suspect this IPLD-ified version will closely resemble the final version used for exchange between compute and requester.

Regretfully, saying RTFM to the users doesn't seem acceptable in 2023!
LOL .. Yes, we need to be user obsessed and improve the user experience of our solution for most of our users. However, if we accommodate every single request, we will end up with an unmaintainable and complex system. Sometimes less is more! So we need to evaluate the long term implications of each feature, understand how many users are asking for it and why, and then decide if the value of introducing it outweigh the added complexity.

I do believe we are on the same page in making our system flexible and allow the requester to do work on behalf of the client to improve the user experience. My feedback is just to evaluate those features more thoroughly


where do you host the sql data?

where do you host the sql data?

Postgres -> https://github.com/caesarHQ/textSQL/blob/a0a390f824d4f8e836766a8461a91bb8014925c3/api/app/config.py#L7


Incorrect behaviour on parsing html

I was trying to pars html content of cnn.com news pages, and when I get body tag, using both find() and  getElementByTag() half the content was gone. I put parsed content into a file, and realized some tags like
 are out of  or  tag, something like this:

  ...
  ...

...

...

...

php code:
<?php

$dom = new PHPHtmlParser\Dom();
$url = 'http://edition.cnn.com/2015/11/19/tennis/world-tour-finals-federer-nishikori/index.html';
$dom->load($url);
file_put_contents('test.html', (string) $dom);

Ah, the code base at the moment only cares about what is in the html tag.... we can most likely solve this by given the option to contain multiple root nodes but at the moment that is not possible.

I am not able to duplicate your issue, please reopen if you are stil experience your issue with more details and I will continue to look into it.


⚠ Bofrost.lu has degraded performance

In 8777395, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1487 ms

Resolved: Bofrost.lu performance has improved in 7af9af7 after 15 minutes.


pypi trouble - ModuleNotFoundError

Tried to run two @pypi-decorated flows:

FractalFlow from https://docs.metaflow.org/scaling/dependencies/libraries
modified Playlist plus, swapping conda for pypi

In both cases, the flow errors out, saying the requested package is not found.  What am I doing wrong?
thanks!
from metaflow import FlowSpec, step, Parameter, conda, conda_base, pypi

def get_python_version():
    """
    A convenience function to get the python version used to run this
    tutorial. This ensures that the conda environment is created with an
    available version of python.

    """
    import platform

    versions = {"2": "2.7.15", "3": "3.9.10"}
    return versions[platform.python_version_tuple()[0]]

class PlayListFlow(FlowSpec):
    """
    The next version of our playlist generator that adds a 'hint' parameter to
    choose a bonus movie closest to the 'hint'.

    The flow performs the following steps:

    1) Load the genre-specific statistics from the MovieStatsFlow.
    2) In parallel branches:
       - A) Build a playlist from the top films in the requested genre.
       - B) Choose a bonus movie that has the closest string edit distance to
         the user supplied hint.
    3) Join the two to create a movie playlist and display it.

    """

    genre = Parameter(
        "genre", help="Filter movies for a particular genre.", default="Sci-Fi"
    )

    hint = Parameter(
        "hint",
        help="Give a hint to the bonus movie algorithm.",
        default="Metaflow Release",
    )

    recommendations = Parameter(
        "recommendations",
        help="The number of movies recommended for the playlist.",
        default=5,
    )

    @step
    def start(self):
        """
        Use the Metaflow client to retrieve the latest successful run from our
        MovieStatsFlow and assign them as data artifacts in this flow.

        """
        # Load the analysis from the MovieStatsFlow.
        from metaflow import Flow, get_metadata

        # Print metadata provider
        print("Using metadata provider: %s" % get_metadata())

        # Load the analysis from the MovieStatsFlow.
        run = Flow("MovieStatsFlow").latest_successful_run
        print("Using analysis from '%s'" % str(run))

        # Get the dataframe from the start step before we sliced into into
        # genre-specific dataframes.
        self.dataframe = run["start"].task.data.dataframe

        # Also grab the summary statistics.
        self.genre_stats = run.data.genre_stats

        # Compute our two recommendation types in parallel.
        self.next(self.bonus_movie, self.genre_movies)

    @pypi(python='3.11.5', packages={"editdistance": "0.6.2"})  # tried 3.9.10 as well
    #@conda(libraries={"editdistance": "0.5.3"})
    @step
    def bonus_movie(self):
        """
        Use the user supplied 'hint' argument to choose a bonus movie that has
        the closest string edit distance to the hint.

        This step uses 'conda' to isolate the environment. Note that the
        package 'editdistance' need not be installed in your python
        environment.
        """

        import sys
        print(sys.executable)
        print(sys.path)

        import editdistance

        # Define a helper function to compute the similarity between two
        # strings.
        def _edit_distance(movie_title):
            return editdistance.eval(self.hint, movie_title)

        # Compute the distance and take the argmin to find the closest title.
        distance = [
            _edit_distance(movie_title) for movie_title in self.dataframe["movie_title"]
        ]
        index = distance.index(min(distance))
        self.bonus = (
            self.dataframe["movie_title"][index],
            self.dataframe["genres"][index],
        )

        self.next(self.join)

    @step
    def genre_movies(self):
        """
        Select the top performing movies from the use specified genre.
        """

        from random import shuffle

        # For the genre of interest, generate a potential playlist using only
        # highest gross box office titles (i.e. those in the last quartile).
        genre = self.genre.lower()
        if genre not in self.genre_stats:
            self.movies = []
        else:
            df = self.genre_stats[genre]["dataframe"]
            quartiles = self.genre_stats[genre]["quartiles"]
            self.movies = [
                df["movie_title"][i]
                for i, g in enumerate(df["gross"])
                if g >= quartiles[-1]
            ]

        # Shuffle the content.
        shuffle(self.movies)

        self.next(self.join)

    @step
    def join(self, inputs):
        """
        Join our parallel branches and merge results.

        """
        self.playlist = inputs.genre_movies.movies
        self.bonus = inputs.bonus_movie.bonus

        self.next(self.end)

    @step
    def end(self):
        """
        This step simply prints out the playlist.

        """
        # Print the playlist.
        print("Playlist for movies in genre '%s'" % self.genre)
        for pick, movie in enumerate(self.playlist, start=1):
            print("Pick %d: '%s'" % (pick, movie))
            if pick >= self.recommendations:
                break

        print("Bonus Pick: '%s' from '%s'" % (self.bonus[0], self.bonus[1]))

if __name__ == "__main__":
    PlayListFlow()

returns
a@AK-Desktop-6600:~/code/testmetaflow/metaflow-tutorials$ python 04-playlist-plus/playlist.py --environment=pypi run
Metaflow 2.10.3 executing PlayListFlow for user:a
Validating your flow...
    The graph looks good!
Running pylint...
    Pylint not found, so extra checks are disabled.
Bootstrapping virtual environment(s) ...
Virtual environment(s) bootstrapped!
2023-10-19 12:58:21.133 Workflow starting (run-id<PHONE_NUMBER>127204):
2023-10-19 12:58:21.151 [1697741901127204/start/1 (pid 6589)] Task is starting.
2023-10-19 12:58:21.428 [1697741901127204/start/1 (pid 6589)] Using metadata provider: local@/home/a/code/testmetaflow
2023-10-19 12:58:21.430 [1697741901127204/start/1 (pid 6589)] Using analysis from 'Run('MovieStatsFlow/1697738002913286')'
2023-10-19 12:58:21.505 [1697741901127204/start/1 (pid 6589)] Task finished successfully.
2023-10-19 12:58:21.508 [1697741901127204/bonus_movie/2 (pid 6592)] Task is starting.
2023-10-19 12:58:21.533 [1697741901127204/genre_movies/3 (pid 6597)] Task is starting.
2023-10-19 12:58:21.860 [1697741901127204/genre_movies/3 (pid 6597)] Task finished successfully.
2023-10-19 12:58:22.059 [1697741901127204/bonus_movie/2 (pid 6592)] /home/a/.pyenv/versions/3.11.5/bin/python
2023-10-19 12:58:22.060 [1697741901127204/bonus_movie/2 (pid 6592)] <flow PlayListFlow step bonus_movie> failed:
2023-10-19 12:58:22.064 [1697741901127204/bonus_movie/2 (pid 6592)] Internal error
2023-10-19 12:58:22.065 [1697741901127204/bonus_movie/2 (pid 6592)] Traceback (most recent call last):
2023-10-19 12:58:22.066 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/cli.py", line 1172, in main
2023-10-19 12:58:22.066 [1697741901127204/bonus_movie/2 (pid 6592)] start(auto_envvar_prefix="METAFLOW", obj=state)
2023-10-19 12:58:22.066 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/_vendor/click/core.py", line 829, in __call__
2023-10-19 12:58:22.066 [1697741901127204/bonus_movie/2 (pid 6592)] return self.main(args, kwargs)
2023-10-19 12:58:22.177 [1697741901127204/bonus_movie/2 (pid 6592)] ^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-19 12:58:22.177 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/_vendor/click/core.py", line 782, in main
2023-10-19 12:58:22.177 [1697741901127204/bonus_movie/2 (pid 6592)] rv = self.invoke(ctx)
2023-10-19 12:58:22.177 [1697741901127204/bonus_movie/2 (pid 6592)] ^^^^^^^^^^^^^^^^
2023-10-19 12:58:22.177 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/_vendor/click/core.py", line 1259, in invoke
2023-10-19 12:58:22.177 [1697741901127204/bonus_movie/2 (pid 6592)] return _process_result(sub_ctx.command.invoke(sub_ctx))
2023-10-19 12:58:22.178 [1697741901127204/bonus_movie/2 (pid 6592)] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-19 12:58:22.178 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/_vendor/click/core.py", line 1066, in invoke
2023-10-19 12:58:22.178 [1697741901127204/bonus_movie/2 (pid 6592)] return ctx.invoke(self.callback, ctx.params)
2023-10-19 12:58:22.178 [1697741901127204/bonus_movie/2 (pid 6592)] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-19 12:58:22.178 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/_vendor/click/core.py", line 610, in invoke
2023-10-19 12:58:22.178 [1697741901127204/bonus_movie/2 (pid 6592)] return callback(args, kwargs)
2023-10-19 12:58:22.179 [1697741901127204/bonus_movie/2 (pid 6592)] ^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-19 12:58:22.179 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/_vendor/click/decorators.py", line 21, in new_func
2023-10-19 12:58:22.179 [1697741901127204/bonus_movie/2 (pid 6592)] return f(get_current_context(), args, kwargs)
2023-10-19 12:58:22.179 [1697741901127204/bonus_movie/2 (pid 6592)] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-19 12:58:22.179 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/cli.py", line 581, in step
2023-10-19 12:58:22.179 [1697741901127204/bonus_movie/2 (pid 6592)] task.run_step(
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/task.py", line 583, in run_step
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] self._exec_step_function(step_func)
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages/metaflow/task.py", line 57, in _exec_step_function
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] step_function()
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] File "/home/a/code/testmetaflow/metaflow-tutorials/04-playlist-plus/playlist.py", line 96, in bonus_movie
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] import editdistance
2023-10-19 12:58:22.180 [1697741901127204/bonus_movie/2 (pid 6592)] ModuleNotFoundError: No module named 'editdistance'
2023-10-19 12:58:22.181 [1697741901127204/bonus_movie/2 (pid 6592)]
**2023-10-19 12:58:22.181 [1697741901127204/bonus_movie/2 (pid 6592)] ['/home/a/code/testmetaflow/metaflow-tutorials/04-playlist-plus', '/home/a/.pyenv/versions/3.11.5/lib/python311.zip', '/home/a/.pyenv/versions/3.11.5/lib/python3.11', '/home/a/.pyenv/versions/3.11.5/lib/python3.11/lib-dynload', '/home/a/.pyenv/versions/3.11.5/lib/python3.11/site-packages']**
2023-10-19 12:58:22.181 [1697741901127204/bonus_movie/2 (pid 6592)] Task failed.
2023-10-19 12:58:22.182 Workflow failed.
2023-10-19 12:58:22.182 Terminating 0 active tasks...
2023-10-19 12:58:22.182 Flushing logs...
    Step failure:
    Step bonus_movie (task-id 2) failed.

Notice the printout for sys.path (marked with **): the micromamba environment that attempts pip install is absent:
Bootstrapping virtual environment(s) ...
    Pip ran into an error while setting up environment:
    command '/home/a/.metaflowconfig/micromamba/bin/micromamba run --prefix /home/a/micromamba/envs/metaflow/linux-64/91bcc23b46dd043 pip3 --disable-pip-version-check --no-input --no-color --isolated install --dry-run --only-binary=:all: --upgrade-strategy=only-if-needed --target=/tmp/tmpzpo2ku8f --report=/tmp/tmpzpo2ku8f/report.json --progress-bar=off --quiet --abi none --abi abi3 --abi cp39 --platform manylinux_2_27_x86_64 --platform manylinux_2_17_x86_64 --platform manylinux_2_20_x86_64 --platform manylinux2010_x86_64 --platform any --platform manylinux_2_26_x86_64 --platform manylinux_2_23_x86_64 --platform manylinux_2_24_x86_64 --platform manylinux2014_x86_64 --platform manylinux_2_18_x86_64 --platform manylinux1_x86_64 --platform manylinux_2_19_x86_64 --platform manylinux_2_25_x86_64 --platform manylinux_2_21_x86_64 --platform linux_x86_64 requests>=2.21.0 editdistance==0.5.3' returned error (1)
    ERROR: Could not find a version that satisfies the requirement editdistance==0.5.3 (from versions: 0.6.0, 0.6.1, 0.6.2)
    ERROR: No matching distribution found for editdistance==0.5.3

(that's why I changed to editdistance==0.6.2)

just found that each execution attempt created a new environment in:
ll ~/micromamba/envs/metaflow/linux-64/
total 44
drwxr-xr-x 11 a a 4096 Oct 19 12:58 ./
drwxr-xr-x  3 a a 4096 Oct 19 12:34 ../
drwxr-xr-x 13 a a 4096 Oct 19 12:58 0d7472272e27b0c/
drwxr-xr-x 13 a a 4096 Oct 19 12:52 21a849a71a17fcd/
drwxr-xr-x 12 a a 4096 Oct 19 12:34 7264f22a78a4409/
drwxr-xr-x 13 a a 4096 Oct 19 12:35 7bd8def62c9d5d5/
drwxr-xr-x 12 a a 4096 Oct 19 12:57 91bcc23b46dd043/
drwxr-xr-x 13 a a 4096 Oct 19 12:39 9db3d6f40db3e76/
drwxr-xr-x 12 a a 4096 Oct 19 12:38 c0dca06c9139377/
drwxr-xr-x 12 a a 4096 Oct 19 12:34 d2bfadb458f041e/
drwxr-xr-x 13 a a 4096 Oct 19 12:48 db23aaf3108b082/
and in 0d7472272e27b0c/lib/python3.9/site-packages/ there is the requested editdistance, but somehow metaflow is not actually using that environment

It's surprising that @pypi is using micromamba pip to install things.  Why not regular pip from the same environment as python executable?  Isn't it possible to steer clear of all serpents?

https://github.com/Netflix/metaflow/pull/1581 related? @savingoyal

@liquidcarbon we are triaging this


Rename accessor method name: rendered_component => rendered_content

Summary
In #1347, rendered component has been renamed to rendered content.
But its method name seems to be the same as before.

Nice work 👏 In addition to this change, I think it would be helpful to update this document as well.
https://github.com/github/view_component/blob/0b55bb7e2c17ecb075789b14fe1a336bd8168344/docs/guide/testing.md


The disable function of the [Bug Report] tree component is not working properly

Element UI version
1.4.5
OS/Browsers version
win7 / Chrome 55
Vue version
2.4.4
Reproduction Link
https://jsfiddle.net/vitorDeng/67zgbzp8/
Steps to reproduce
无
What is Expected?
...
What is actually happening?
无

Fixed in #7226


Beta broken 2

yea

Please don't open useless pull requests. The two broken branches are broken due to circular imports causing final compilation to not be in correct synchronous order.

hey sorry lol just testing stuff
On Sun, 1 May 2022 at 10:08, Nobu @.***> wrote:

Please don't open useless pull requests. The two broken branches are
broken due to circular imports causing final compilation to not be in
correct synchronous order.
—
Reply to this email directly, view it on GitHub
https://github.com/MCBE-Utilities/BeAPI/pull/40#issuecomment-1114076608,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYNVUIQHVZRMMLYLTD2RZGLVHXDPRANCNFSM5UY2CSSA
.
You are receiving this because you authored the thread.Message ID:
@.***>


Get rid of the little menus that pop up with each tool

The menu should be part of the toolbar; not over the drawing

Let's at least fix the cursor preview being in a random place


Fix typos in comments, log messages and strings

List of fixed words:

successful
should
revocable
NuGet
production
pre-calculated
failed
truly
because
unknown
package
PowerShell
addresses
configuration
maintenance
perform
accepted
indices
recursive
uppercase
broken
purposes
empty

Hi @alexandear, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla2.dotnetfoundation.org.

TTYL, DNFBOT;

@alexandear, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, DNFBOT;

LGTM. @scottbommarito , want to add another pair of eyez?

Looks good.


[FEATURE REQ]

Please describe the feature.
If customer has created 8 revisions for an API on his dev env APIM instance but only 1 is made current after thorough testing of all 8 revisions. Now he wants to promote this APIM to production env.
There is no way available so that he gets only the final revision (current one) deployed in prod env but not rest 7 revisions.
If he follows CI/CD pipeline all code will be deployed including rest 7 revisions. Same is the case for APIOps.
Customer is looking for option to select revisions or only current revision should move to next env or prod env.

Hi @ankisingh243,
In APIM, revisions can (almost) be treated as distinct APIs with different names. When you extract, you should have something like this structure:

artifacts/apis/api
artifacts/apis/api;rev={1}
artifacts/apis/api;rev={2}

If you want to prevent rev1 and rev2 from being promoted, here are a few options:

Delete the artifacts/apis/api;rev={1} and artifacts/apis/api;rev={2} folders before promoting.
Use .gitignore to prevent those folders from being added to source control.

I will close this as there is no further work planned on this feature.


Error get key by certificate

ProviderMicrosoft::getKey not return key by certificate. cannot search container

Work nice only for GOST key with CryptoPro CSP.


System information page on start page

Hi,
Firstly i don’t know how bad this idea is to ask.
I am thinking to change the first page i.e. System Information page to some other module as per my wish.That means whenever i login to the webmin the first page to show would be my module.
Can this possible to do?If any chance of achieving this please let me know how it would be?
Thanks in advance.

Hi,
Goto Webmin->Webmin Configuration-> Index Page Options and set After login, always go to module to whatever you wish.

thanks a lot. :+1:

Don't mention it! :wink:

That is the least we can do it for your extraordinary work.. :)


Self closing slot tag get red in child component

here is my parent component:
<Modal>     <slot>im inside parent here we go</slot> </Modal>
and here is my child component:
<template>     <slot /> </template>
here in child component the slash got red unexpectedly.

see https://github.com/vuejs/language-tools/issues/1317
btw self close tag does not 100% equal to normal tag like
 !==

 not 100% sure about slot tho

Duplicate of #1317


Add convolutional encoder and highway layer

Description

Add convolutional encoder
Add highway layer and higway bias initializer
Corresponding docs

Checklist
Essentials

[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage
[ ] Code is well-documented

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

Job PR-129/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-129/2/index.html

Job PR-129/13 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-129/13/index.html

Job PR-129/15 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-129/15/index.html

Job PR-129/17 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-129/17/index.html

Job PR-129/19 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-129/19/index.html

Job PR-129/20 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-129/20/index.html


NIFI-7870 - Added an allow list for static resources so that they can…

… be accessed anonymously.
NIFI-7870 - Fixed test verbiage.
NIFI-7870 - Added license headers. Checkstyle fix.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Enables X functionality; fixes bug NIFI-YYYY.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically main)?

[x] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[x] Have you verified that the full build is successful on JDK 8?
[x] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

Interested in feedback on this one, let me know if you think the allow list is too long, and if the checking that I'm doing against the list is sufficiently robust.

I feel like maintaining this list is going to be very challenging, how would anyone know to update this?
The most correct solution to me would be for us to use a Cookie for the interacts between NiFi UI and the REST API, this way the credentials are submitted on all requests, similar to how a client certificate is automatically submitted. We would still want to support the current Authorization header approach for direct API access.
If we have a good reason for not doing that, and we still want to pursue some type of allow list, could it be done with some kind of patterns? Not sure if all the front-end stuff comes from some base path that we could just allow anything that starts with the base path, or maybe use extensions and say that we allow any .html, .css, .js, etc.

I tend to agree. Maybe we can discuss using a cookie in combination with an authorization header to avoid breaking backwards compatibility but getting the benefit of using the cookie. This would, I think, eventually remove the need for a few different parts of code like the OTP for downloads and perhaps others where we've had to do a bit of extra work to use an authorization header.

I think the plan here was that, for the bundled advanced UIs, they could exist in this list and be granted anonymous access. For custom advanced UIs that the community may develop/use, they will need to be accessed anonymously by enabling the "nifi.security.allow.anonymous.authentication" property. This reduces the list maintenance requirement a little bit, but does not eliminate it. It's possible that the list could be cut down by my manually looking at the resources that need to be allowed, in the UI, to check what is not accessible and enabling only those. From memory it was at least a few CSS files that caused the bigger issues. As far as wildcarding some of this, it was recommended to be explicit on the file name and not to use patterns. Maybe we can get @alopresto to chime in and see what he thinks of my approach to this PR.
Having said that, I do believe that moving towards using a cookie for storing the JWT in the browser may be a reasonable option. I've been researching the benefits vs risks, the details of which I'm still collating. It would eliminate a few parts of code that we've so far had to implement as a side effect of using an explicit authorization header.

Concurring with @bbende that maintaining a static list of allowed files seems very brittle.  At minimum, using a much smaller set of regular expression patterns would be easier to maintain.  If the goal is to avoid pattern matching at runtime, perhaps introducing something in the build process to find static resources and generate a list could be considered.

Closing this PR with the goal of creating a better solution.


Build system: make launch errors in completely untouched classes

Hi guys,
I have been through quite a hustle trying to implement an own radar sensor and I wanted to share my story with you.

I looked at the sensor tutorial and did the same steps but created my own class "AGroundTruthRadar". Outcome: PythonAPI compiles without errors, launch doesn't throw errors in my files but in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp.
I took the sensor tutorial and only changed the files' contents to my needs. My sensor remains named "ASafeDistanceSensor" for pure belief in magic. Outcome: PythonAPI compiles without errors, launch compiles without errors.
I change the class name to "AGroundTruthRadar". Outcome: PythonAPI compiles without errors, launch doesn't throw errors in my files but in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp.
I freak out and change the name back to "ASafeDistanceSensor". Outcome: PythonAPI compiles without errors, launch doesn't throw errors in my files but in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp.
I try compiling with the same source code on another machine. Outcome: PythonAPI compiles without errors, launch doesn't throw errors in my files but in "carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla\Util\BoundingBoxCalculator.cpp".

Please tell me there is a solution for this, @bernatx @germanros1987...
Best regards
Robert

I've encountered a similar issue, I believe the root cause is Carla's use of unity builds. see https://github.com/carla-simulator/carla/issues/4225 for more details.
unity builds may hide missing #include declarations, since multiple .cpp files are concatenated into a single .cpp file by the build system. Adding new files changes the grouping of .cpp files in the build, and thus can "uncover" those missing #include declarations.
The current workaround we've found is to add the missing #include's manually, whenever we encounter compilation errors.

I am facing the same issue after creating a new c++ actor
E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(28): error C2027: use of undefined type 'carla::road::element::RoadInfoSignal'   E:\develop\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla\Traffic\SignComponent.h(23): note: see declaration of 'carla::road::element::RoadInfoSignal'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(28): error C2530: 'validity': references must be initialized   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(28): error C3531: 'validity': a symbol whose type contains 'auto' must have an initializer   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(28): error C2143: syntax error: missing ';' before ':'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(28): error C2143: syntax error: missing ';' before ')'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(30): error C2660: 'carla::geom::Math::GenerateRange': function does not take 1 arguments   E:\develop\carla\Unreal\CarlaUE4\Plugins\Carla\CarlaDependencies\include\carla/geom/Math.h(124): note: see declaration of 'carla::geom::Math::GenerateRange'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(30): error C3531: 'lane': a symbol whose type contains 'auto' must have an initializer   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(30): error C2143: syntax error: missing ';' before ':'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(30): error C2660: 'carla::geom::Math::GenerateRange': function does not take 1 arguments   E:\develop\carla\Unreal\CarlaUE4\Plugins\Carla\CarlaDependencies\include\carla/geom/Math.h(124): note: see declaration of 'carla::geom::Math::GenerateRange'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(30): error C2143: syntax error: missing ';' before ')'   E:/develop/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp(36): error C2027: use of undefined type 'carla::road::element::RoadInfoSignal'   E:\develop\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla\Traffic\SignComponent.h(23): note: see declaration of 'carla::road::element::RoadInfoSignal'

Hi, I can confirm that currently the most promising approach is adding all the missing includes. It took me some time, but in the end it worked.

Fixed by adding missing includes in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/YieldSignComponent.cpp
#include "YieldSignComponent.h"
#include "TrafficLightState.h"
#include "Carla/Vehicle/CarlaWheeledVehicle.h"
#include <queue>

#include <compiler/disable-ue4-macros.h>
#include <carla/road/element/RoadInfoSpeed.h>
#include <carla/road/element/RoadInfoSignal.h>
#include <compiler/enable-ue4-macros.h>

/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/carla/rpc/ActorAttributeType.h:11:17: error: namespaces can only be defined in global or namespace scope
namespace carla {
                ^
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Carla/Module.Carla.3_of_3.cpp:4:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Trigger/FrictionTrigger.cpp:6:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Trigger/FrictionTrigger.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Game/CarlaEpisode.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorDispatcher.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorDefinition.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:13:
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Inc/Carla/ActorAttribute.generated.h:21:1: error: expected expression
template<> CARLA_API UScriptStruct* StaticStruct<struct FActorAttribute>();
^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Inc/Carla/ActorAttribute.generated.h:28:1: error: expected expression
template<> CARLA_API UScriptStruct* StaticStruct<struct FActorVariation>();
^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Inc/Carla/ActorAttribute.generated.h:45:1: error: expected expression
template<> CARLA_API UEnum* StaticEnum<EActorAttributeType>();
^
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Carla/Module.Carla.3_of_3.cpp:4:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Trigger/FrictionTrigger.cpp:6:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Trigger/FrictionTrigger.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Game/CarlaEpisode.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorDispatcher.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorDefinition.h:9:
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:22:15: error: no member named 'ActorAttributeType' in namespace 'carla::rpc'; did you mean 'EActorAttributeType'?
  Bool      = CARLA_ENUM_FROM_RPC(Bool)      UMETA(DisplayName = "Bool"),
              ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:51: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ^~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:19:12: note: 'EActorAttributeType' declared here
enum class EActorAttributeType : uint8
           ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:22:35: error: no member named 'Bool' in 'EActorAttributeType'
  Bool      = CARLA_ENUM_FROM_RPC(Bool)      UMETA(DisplayName = "Bool"),
              ~~~~~~~~~~~~~~~~~~~~^~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:84: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:23:15: error: no member named 'ActorAttributeType' in namespace 'carla::rpc'; did you mean 'EActorAttributeType'?
  Int       = CARLA_ENUM_FROM_RPC(Int)       UMETA(DisplayName = "Integer"),
              ^~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:51: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ^~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:19:12: note: 'EActorAttributeType' declared here
enum class EActorAttributeType : uint8
           ^
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Carla/Module.Carla.2_of_3.cpp:21:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Recorder/CarlaReplayerHelper.cpp:20:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/TrafficLightBase.h:12:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/TrafficLightComponent.h:10:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Traffic/SignComponent.h:12:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/OpenDrive/OpenDrive.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/OpenDrive/OpenDriveMap.h:10:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/carla/road/Map.h:10:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/carla/geom/Rtree.h:11:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry.hpp:17:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry/geometry.hpp:49:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry/strategies/strategies.hpp:115:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry/strategies/geographic/distance_cross_track.hpp:37:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry/strategies/geographic/intersection.hpp:28:
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry/formulas/sjoberg_intersection.hpp:568:18: warning: declaration shadows a field of 'sjoberg_geodesic<CT, Order>' [-Wshadow]
        CT const asin_tj_t0j = asin(Cj * tan_betaj / sqrt_1_Cj_sqr);
                 ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/CarlaDependencies/include/boost/geometry/formulas/sjoberg_intersection.hpp:606:8: note: previous declaration is here
    CT asin_tj_t0j;
       ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:23:15: error: no member named 'Int' in 'EActorAttributeType'; did you mean 'EFormatArgumentType::Int'?
  Int       = CARLA_ENUM_FROM_RPC(Int)       UMETA(DisplayName = "Integer"),
              ^~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:51: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/UE4.26/Engine/Source/Runtime/Core/Public/Internationalization/Text.h:138:3: note: 'EFormatArgumentType::Int' declared here
                Int,
                ^
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/Carla/Module.Carla.3_of_3.cpp:4:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Trigger/FrictionTrigger.cpp:6:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Trigger/FrictionTrigger.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Game/CarlaEpisode.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorDispatcher.h:9:
In file included from /home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorDefinition.h:9:
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:24:15: error: no member named 'ActorAttributeType' in namespace 'carla::rpc'; did you mean 'EActorAttributeType'?
  Float     = CARLA_ENUM_FROM_RPC(Float)     UMETA(DisplayName = "Float"),
              ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:51: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ^~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:19:12: note: 'EActorAttributeType' declared here
enum class EActorAttributeType : uint8
           ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:24:35: error: no member named 'Float' in 'EActorAttributeType'
  Float     = CARLA_ENUM_FROM_RPC(Float)     UMETA(DisplayName = "Float"),
              ~~~~~~~~~~~~~~~~~~~~^~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:84: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:25:15: error: no member named 'ActorAttributeType' in namespace 'carla::rpc'; did you mean 'EActorAttributeType'?
  String    = CARLA_ENUM_FROM_RPC(String)    UMETA(DisplayName = "String"),
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:51: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ^~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:19:12: note: 'EActorAttributeType' declared here
enum class EActorAttributeType : uint8
           ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:25:35: error: no member named 'String' in 'EActorAttributeType'
  String    = CARLA_ENUM_FROM_RPC(String)    UMETA(DisplayName = "String"),
              ~~~~~~~~~~~~~~~~~~~~^~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:84: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:26:15: error: no member named 'ActorAttributeType' in namespace 'carla::rpc'; did you mean 'EActorAttributeType'?
  RGBColor  = CARLA_ENUM_FROM_RPC(RGBColor)  UMETA(DisplayName = "RGB Color (comma separated)"),
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorAttribute.h:15:51: note: expanded from macro 'CARLA_ENUM_FROM_RPC'
#define CARLA_ENUM_FROM_RPC(e) static_cast<uint8>(carla::rpc::ActorAttributeType:: e)
                                                  ^~~~~~~~~~~~

I am also facing this issue. I have created my own sensor model in carla and  didn't touch any other files, still I am getting this error. Did anyone faced this issue?

@benraziel can you please help me with this problem?


Advanced visualization levels

As HELM covers all 4 levels from atoms over monomers, simple polymers to complex polymers it would be extremely helpful if the view within the HELM Editor could be switched between these 4 levels.  In details this means (1) an additional view on the atom level (ideally only of the selected residues and bonds!) directly in the HELM editor instead of copy/paste of the SMILES or MOL files to other external applications.
(2) In addition a high-level schematic view of the complex polymer level (ideally using the well known standard shape of antibodies where possible) would abstract a molecule comprising several simple polymers into a simple but yet correct schema. This  would provide both a great overview to selectively zoom into deeper levels where required and a high-level schema (image) to be stored in databases for fast orientation through a list of entries.

(1) already implemented in HELM Editor
(2) solved on antibody level by HELM Antibody Editor