Detect violations when using React.forwardRef

Hi!
I just found out about this package recently. Awesome idea.
I'm adding it to a project right now, and I noticed that it doesn't detect violations inside React.forwardRef.
Is it possible to add support for that?

It was pretty easy to figure out. Made a PR.
Please let me know if it's OK!


Timed blockage

|t would be great if we could schedule how long a page is blocked.
For instance, some users want to block email during the morning, and some users might want to have access to certain websites only during certain hours of the day.

I guess having per-site scheduled blocking could be too much to manage. How about all sites share the same scheduled blocking?

I was jus thinking about this.
Something like blocking the websites on weekdays, but not on weekends?

Indeed, any scheduling is appreciated. 🙌

I guess having per-site scheduled blocking could be too much to manage. How about all sites share the same scheduled blocking?

This would work perfect for my use case. 8am-6pm I want my sites blocked, after I don't!

@visch Sounds good. Shall we have option to set different times for workdays and weekends?
I think this could be best to have:
Button to add timed blockage, where you could click to toggle any day (S, M, T, W, T, F, S), and specify time (from, to). You could have more rules (to address workdays or weekends or as you like), you could modify them, delete them, add new.
If that sounds ok, I think we could have it like that. I can then start working on it, so we have it in next version.

@visch Sounds good. Shall we have option to set different times for workdays and weekends? I think this could be best to have: Button to add timed blockage, where you could click to toggle any day (S, M, T, W, T, F, S), and specify time (from, to). You could have more rules (to address workdays or weekends or as you like), you could modify them, delete them, add new. If that sounds ok, I think we could have it like that. I can then start working on it, so we have it in next version.

@visch Sounds good. Shall we have option to set different times for workdays and weekends? I think this could be best to have: Button to add timed blockage, where you could click to toggle any day (S, M, T, W, T, F, S), and specify time (from, to). You could have more rules (to address workdays or weekends or as you like), you could modify them, delete them, add new. If that sounds ok, I think we could have it like that. I can then start working on it, so we have it in next version.

That sounds perfect to me!

Great! I am on it. Will be in next version in about 7 days.

There's some conflict:
Context menu (used to "Block this site"), which we recently added (version 5.0), is visible when Block Site is Enabled.
Timed blockage, should block sites only during specified days and hours. That means, "Block this site" action from the Context menu, if outside Timed blockage, could still add the site to the blocked list, but definitely would NOT block the site right away (as it does now), and that could look broken (I requested to Block this site, but nothing happened).
Any ideas how to approach this?

I have a suggestion to enhance our blocking feature. Could we add a cooldown period option? This would allow users to pause all blocks for 10, 30, or 60 minutes with a button click, giving them more control and flexibility.

There's some conflict: Context menu (used to "Block this site"), which we recently added (version 5.0), is visible when Block Site is Enabled. Timed blockage, should block sites only during specified days and hours. That means, "Block this site" action from the Context menu, if outside Timed blockage, could still add the site to the blocked list, but definitely would NOT block the site right away (as it does now), and that could look broken (I requested to Block this site, but nothing happened). Any ideas how to approach this?

Maybe, a sub-dropdown can help navigate this UX problem, i.e 1> Add to Blocklist 2> Enter / Exit Cooldown


Add support for quick help in monaco-json

I would like to add support for quick help in JSON language support if a JSON schema is provided for the JSON file the user is editing. Here is an example: https://cl.ly/53d348dd39a4
It's possible to implement it using overlay widgets but the language support doesn't work on the main thread and instead runs under the LSP. Ideally, we can send a message from the WebWorker to the main thread and create an overlay widget for the editors if a flag is enabled.
The main problem is that the language support doesn't really have an access to the editor, I can work on this feature and create a PR on monaco-json if there's interest, I already have a prototype but couldn't really find a way to make this feature embedded into monaco-json as we don't have access to the created editors. Instead, I'm creating an overlay widget in my editor separately talking to the language server.

fyi @aeschli

IMO adding to monaco-json is not the right approach as this is quite a specific feature. Instead you could be looking into having your own monaco-json and replace the original one.

@aeschli thanks for the answer. What would be the right approach to implement such a feature based on my previous message?

@buremba Sorry, can you be more specific what information you need?

I forked monaco-json and implemented getSchema(uri: Uri, position: Position)``monaco.languages.json.getWorker() and implemented an overlay widget using json-schema-visualizer. It looks to be the convenient way for this use-case. Thanks!


IExcelDataReader.AsDataSet() does not contains definition for AsDataSet

Installed via nuget ExcelDataReader version 3.6

Hi,
Quoting the README
Install the ExcelDataReader.DataSet extension package to use the AsDataSet() method


Loading view snaps underneath navigation bar

For some reason, the view is snapping underneath my navigation bar? I have used the demo code given in the ReadMe.
// Initialize tableView
let loadingView = DGElasticPullToRefreshLoadingViewCircle()
loadingView.tintColor = UIColor(red: 78/255.0, green: 221/255.0, blue: 200/255.0, alpha: 1.0)
tableView.dg_addPullToRefreshWithActionHandler({ [weak self] () -> Void in
    // Add your logic here
    // Do not forget to call dg_stopLoading() at the end
    self?.tableView.dg_stopLoading()
}, loadingView: loadingView)
tableView.dg_setPullToRefreshFillColor(UIColor(red: 57/255.0, green: 67/255.0, blue: 89/255.0, alpha: 1.0))
tableView.dg_setPullToRefreshBackgroundColor(tableView.backgroundColor!)

Video below..

By setting navigationController?.navigationBar.translucent = false, I have got to this point.

Now I can't seem to get it working properly where it only shows the loading view:

After pull to refresh
While loading

It seems the loading view is showing always and even when stopLoading() is called, the cell remains?

Fixed by removing the following code... 😄
override func viewDidLayoutSubviews() {
    if let rect = self.navigationController?.navigationBar.frame, bottomRect =  self.tabBarController?.tabBar.frame {
            let y = rect.size.height + rect.origin.y
            let y2 = bottomRect.size.height
            self.tableView.contentInset = UIEdgeInsetsMake( y, 0, y2, 0)
    }
}


Optimize images

Using the images provided by: https://developers.google.com/speed/pagespeed/insights/?url=local.hackcu.org

Merge whenever you finish compressing all images!

I only compressed the ones that are visible on the site. We can compress the rest when needed. Also, the ones that Google Requested

Ok. Merging now.


add __cxa_thread_atexit_impl on freebsd

Fixes https://github.com/rust-lang/miri/issues/3276

@bors r+

:pushpin: Commit 90ae4609ba9f3f0eeb7f7bcda10c48dc0e466cfa has been approved by RalfJung
It is now in the queue for this repository.

:hourglass: Testing commit 90ae4609ba9f3f0eeb7f7bcda10c48dc0e466cfa with merge ebff47be353d9e850b6246c30ca3e5bb3dabfbab...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing ebff47be353d9e850b6246c30ca3e5bb3dabfbab to master...


New Theme: Notepadium

Theme submission
Please make sure that you have read the theme submission guidelines before submitting a theme. The guidelines provide all relevant information and requirements that have to be fulfilled before the submission. Please tick the relevant boxes for your theme in the checklist below:
Link to my theme repository: https://github.com/cntrump/hugo-notepadium
I made sure that...

[x] the repository contains a good README.md describing my theme
[x] an open source license has been added to LICENSE.md
[x] all metadata have been added to theme.toml
[x] screenshots have been added in the images/ folder with the required dimensions
[x] in case I'm using a customized demo via the exampleSite folder that

[x] https://example.com is set as base url in exampleSite/config.{toml, yaml, json} to avoid the abuse of unused domains
[x] I tested that my theme's demo works with the content directory of gohugoio/HugoBasicExample

[x] I tested my theme against the gohugoio/HugoBasicExample

[x] I've checked the developer tools' console in my browser for error messages

[ ] in case my theme is using Hugo Pipes features like toCSS and PostCSS that I have committed the /resources directory with all generated assets, for my theme to work in the basic version of Hugo

N.B. By submitting a theme to the Hugo Themes Showcase you understand that you need to maintain your theme. You will need to keep an eye on whether your theme's demo generates on the Hugo website, whenever there is a new release. We are a very small team and we just cannot keep up opening issues in the various theme repositories to notify authors. Whenever a theme demo breaks and remains broken then at some point it will be removed from the list without prior warning. If you no longer wish to maintain a theme please let us know.
Feel free to ask questions. We're glad to help.

Hello @cntrump,
thank you for submitting your theme :+1: Let's start with the review:

please use this url to link the screenshot at the top of the README. Relative paths work on Github but not on www.themes.gohugo.io.
for security reasons all demos use the content provided by the hugoBasicExample repository. It also provides an about page that is accessible under /about. Would it be possible to change the link in your config file from /pages/about to about to avoid a 404 error?

Hello @cntrump,
thank you for submitting your theme to us 👍 Let's start with the review:

please use this url to link the screenshot at the top of the README. Relative paths work on Github but not on www.themes.gohugo.io.
for security reasons all demos use the content provided by the hugoBasicExample repository. It also provides an about page that is accessible under /about. Would it be possible to change the link in your config file from /pages/about to about to avoid a 404 error?

Thanks for review.
I fixed:

[x] screenshot updated
[x] replaced /exampleSite/content with hugoBasicExample/content

live demo for exampleSite: https://examplesite.lvv.me/

Please use this absolute url to link the screenshot at the top of your theme's README. Relative paths don't work on www.themes.gohugo.io
Otherwise your theme looks fine :+1:

Your theme should appear soon on Hugo's theme site. Next, I'll promote your theme on Hugo's official Twitter account.

Your theme should appear soon on Hugo's theme site. Next, I'll promote your theme on Hugo's official Twitter account.

Thanks! 😁


ADO: Improved Setup Error Handling

What is changing?
The purpose of these changes are to give the users a better experience when they encounter errors during the setup script by:

Adding the ability to skip creating a project if it already exists.  This is useful if a user encountered a error after the project was created and reran the command.

@j-dunham ➜ /workspaces/gai-labs (main) $ ./azure_devops/bootstrap/setup --organization $org --project ado-lab --access-token $pat
Project 'ado-lab' already exists. Would you like to continue? [y/N]y
Extracting assets...
Creating repository ado-lab...

Customizing the error message based on the ADO API status code and using abort to prevent the stack trace from making the error hard to read

Example of a bad token
Error encountered on request:https://dev.azure.com/jd-testing-org/_apis/projects?api-version=6.0
  StatusCode:302
  Message: 'Please check your Azure DevOps access token is correctly entered.'

Example of bad permissions
Error encountered on request:https://dev.azure.com/jd-testing-org/_apis/projects?api-version=6.0
  StatusCode:401
  Message: 'Please check your Azure DevOps access token has the correct permissions.'

How was this tested
👀
Closes github/valet#4974

Are these revisions too wordy?

Please ensure that your Azure DevOps access token is entered correctly.
Please ensure that your Azure DevOps access token has the correct permissions.

Are these revisions too wordy?

Please ensure that your Azure DevOps access token is entered correctly.
Please ensure that your Azure DevOps access token has the correct permissions.

I don't think so.  Looks good to me!


Balance for zone

how about add some description

how about add some description

emmm.. It will be a long story


Search results are not appearing as expected

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/1626
Originally created by @bencodezen on 2020-01-28 21:27:49

Nicole created a dashboard where she was trying to combine two filter labels and getting odd results.
More context to be added by Nicole.

View 5 previous comments from the original issue on GitLab


Chage Data Grid Timestamp format for Fb4 compatibility

The current format of timestamp is D.N.Y, H:M:S.T, but for Fb4 this format returns a exception in execution, so I'll remove the comma from the default format, new format:  D.N.Y, H:M:S.T. @mariuz any problem with it?
Read<EMAIL_ADDRESS>Thanks

My sugesttion is use yyyy-MM-dd HH:mm:ss.t for timestamp format.
Best Regards,
Luciano

My concern is not to change a lot from the current format, because of this I suggest just to remove the comma

You're right @arvanus ! In my opinion, we should focus first on full support for Firebird 3 and then think about support for Firebird 4.
Best Regards,
Luciano

You're right @arvanus ! In my opinion, we should focus first on full support for Firebird 3 and then think about support for Firebird 4.
Best Regards,
Luciano
I agree


License

Good morning,
I saw you use tensorflow.
I would like to know if I can use this library (face-api and tensorflow) for commercial use.
Thank you.

face-api.js has a MIT license and is built on top of tfjs-core, which comes with the Apache Version 2.0 license. So yes you can use it for commercial purposes.


[WIP] Change base widget used to position install plugin pop up message/warning and delta

References and relevant issues
Fixes #65
Description
Checking the code related with the install plugin pop up positioning, seems like the logic was using the process error indicator widget (one of the labels used to show the actions state on the bottom rigth side of the dialog). Since this label is only shown when there is a code error when executing an installation, using it as the base one to get the position where the pop up should appear causes the pop up to end up using as base position the top left dialog when no error happens (since the label is hidden and its position gets mapped to that base point)
Following the above this PR changes the base widget used to get the base position where the pop up should be shown. Some exploration about what widget to use is being done:

Option
Preview

Use the close button (which always is visible), and use a -50 delta to change the base coordinates to show the pop up along side the label showing the status of the actions (✔  or ⚠)

(Current PR state) Use the installed plugins counter label using the label text width + 5 to calculate the initial position of the pop up

Notes

Maybe is worthy to define a better place to put the popup?

Thanks! Is there a risk that if the base window is attached to the right side of the display, the popup will be hidden beyond the edge?

Is there a risk that if the base window is attached to the right side of the display, the popup will be hidden beyond the edge?

If I'm understanding correctly the case mentioned I would say yes. So, for example, if I put the dialog to touch the screen right edge I see something like:

Could it be better to position the popup in a place that ensures it will be inside the dialog? Maybe along side the installed plugins counter label? So something like:

Let me know!

Improving the way feedback is given after installation with a new widget that better shows status of the dialog operations or using the already available napari notifications component makes sense to me (indeed the current approach to show notifications/info after installing/before uninstalling is easy to confuse with a tooltip). The only thing on the top of my head that makes me prefer the idea of a message bar widget inside the plugin dialog is the possibility of not being aware of the notification from the main napari window due to the plugin manager dialog being bigger than the main window. So for example in the following situation:

It would be easy for an user to be unaware that something is being shown over the application main window (unless they first resize the plugin manager dialog or make the main window bigger than the dialog).
Maybe asking for an opinion about this to a designer (e.g Isabela) could be worthy? 🤔

Given the upcoming napari release, maybe going forward with the position change fix of the pop up proposed on this PR could be good enough? And maybe could be worthy to open a new issue to discuss more in dept improvements/new ideas mentioned here related to the way users get feedback for install/unistall actions for future work @jaimergp ?

Sounds good.


why not using java generics?

if you inherit from ArrayList, why not extending ArrayList ? to be more type-safe?

The library was written for Java 2... according to the google page at least. Generics came into Java with version 5... I think. I copied this library and polished it a bit.
https://github.com/RalleYTN/SimpleJSON


Get rid of pprint.hpp

pprint.hpp is a cool project that I'm barely using here, and there's no real reason to have another header that's twice the size of cpp-nbt when I could write a little stub of a printer to do the same job.

That was way easier than I though it was going to be


Report package name and cwd during script errors

Revisit of #102, but against rebirth. This is nearly a trivial change since rebirth appears to already have everything needed in context :sunglasses:. Not really sure how to test this! Suggestions?

Added some comments. Appreciate the PR :+1:

I was revisiting this tonight, have a few questions (above). Seems like most of this is in the codebase already! But I still think it would be useful to log the name of the package in the case of an error, unless I'm misunderstanding how this code is executed.

This is already in the code base.


IEEE 1588 default profile missing

IEEE 1588 defines a default profile (annex J) which is supported by some devices.
The enumeration types representing profiles (e.g. ptp-profiles) do not have a value for this profile. They need to be extended if this profile is of interest.

Proposal to the 5G-xhaul call on 7th of September 20022:

The existing PtpProfiles and TimeSyncProfiles enumerations shall be complemented by the additional value "IEEE1588".
The additional value shall hold the comment: "IEEE1588 default profile according to annex J."
The existing TimeSyncProfilesEnabler enumeration shall be complemented by the additional value "ENABLE_IEEE1588".

I agree to this two points.

Decision made at the 5G-xhaul call on 21st of September 20022:

The existing PtpSync_Pac::PtpProfiles and NeSync_Pac::TimeSyncProfiles enumerations shall be complemented by the additional value "IEEE1588".
The additional value shall hold the comment: "IEEE1588 default profile according to annex J."
The existing NeSync_Pac::TimeSyncProfilesEnabler enumeration shall be complemented by the additional value "ENABLE_IEEE1588".


Problem with JSON output format

Problem
Thanks for fixing the problem about the JSON output option crashing the flow-cli, but now I think we have another problem.
When I try to return an array of struct data, now all the struct is not properly formatted and it's impossible to parse it.
Basically what you're returning now it's not a real JSON anymore.
For example this is what I get when I query for a list of test data
{"result":"[A.f8d6e0586b0a20c7.Test.TestData(id: 0, name: \"test\", ownerName: \"test\", ownerAddress: 0xf8d6e0586b0a20c7, description: \"test test\", testBool: false, testInt: 3, testFix:<PHONE_NUMBER>.00000000)]"}
Steps to Reproduce
Try to output any script as JSON and parse it with PHP or any other tool
Acceptance Criteria
If I'm able to parse the JSON output of the flow-cli
Context
Using the latest flow-cli released today on macOS
Thanks!!

One more thing, same thing happens with events too.
Transactions seems to be fine.

And just in case you missed it, the Address field type is not within quotes, so it won't be parsed as string but should be in my opinion

Hey @luca-mayven thanks for reporting! I've confirmed that this is a bug.
We'll work on a fix and post any updates here 😄

Thanks a lot Pete!!


sickbeard_mp4_automator doesn't include all audio tracks despite the configuration

Hi! I have configured sickbeard_mp4_automator to include all original audio tracks, but it doesn't work as expected.
The original mkv has multichannel audio in FR language.
As a bonus, I would like to be able to create stereo tracks of all original audio tracks.
I will appreciate any help. Thanks in advance
[Audio]
codec = aac, libfdk_aac, libfaac
languages =
default-language =
include-original-language = True
first-stream-of-language = False
channel-bitrate = 128
variable-bitrate = 0
max-bitrate = 0
max-channels = 2
filter =
profile =
force-filter = False
sample-rates =
sample-format =
copy-original = True
aac-adtstoasc = False
ignored-dispositions =
force-default = False
unique-dispositions = False
stream-codec-combinations =
[Audio.Sorting]
sorting = language, channels.d, map, d.comment
default-sorting = channels.d, map, d.comment
codecs =
[Universal Audio]
codec = aac
channel-bitrate = 128
variable-bitrate = 0
first-stream-only = False
filter =
profile =
force-filter = False

managed


Logo Design

Hi @Gauravv97 , I'm a graphic designer after I review your project, The current logo looks simple and less relevant. I want to create an elegant and modern logo. And it's free for you. What do you think?

Hi @zularizal  I created LNotes as a college project, I wanted it to be open source so everyone can use and contribute to it. I am glad that you reviewed it and shown interest in LNotes. Feel free to contribute in any way you like.
I designed this logo by putting objects here and there, I believe any logo would be better than this one. So if your logo does end up on LNotes I will surely give you appropriate credit.

Hi @Gauravv97 , this is my design logo for your project. This logo is inspired by the scan icon, paper icon and initial LN from the word LNotes. What do you think? And what about the icon, A or B?
If something wants to be changed, let me know.

Thanks for creating such wonderful artworks. I like logo B a little bit more than A. There are some changes i would like to see for comparison, can you share your mail id so i can reach you and explain some ideas and changes.

Of course, you can contact me at this email<EMAIL_ADDRESS>

I will surely do that, thanks for your interest @zularizal .


Error while forecasting using processed GFS data in Autoregressive rollout code

I'm attempting to use processed GFS (Global Forecast System) data for forecasting. The processed data's coordinates, dimensions, and variables are nearly identical to the example data provided. However, when running the Autoregressive rollout code snippet, specifically this segment:
# Autoregressive rollout code snippet
assert model_config.resolution in (0, 360. / eval_inputs.sizes["lon"]), (
  "Model resolution doesn't match the data resolution. You likely want to "
  "re-filter the dataset list, and download the correct data.")

print("Inputs:  ", eval_inputs.dims.mapping)
print("Targets: ", eval_targets.dims.mapping)
print("Forcings:", eval_forcings.dims.mapping)

predictions = rollout.chunked_prediction(
    run_forward_jitted,
    rng=jax.random.PRNGKey(0),
    inputs=eval_inputs,
    targets_template=eval_targets * np.nan,
    forcings=eval_forcings)
predictions

I encounter an error message,The error message is as follows.
File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/rollout.py:69, in chunked_prediction(predictor_fn, rng, inputs, targets_template, forcings, num_steps_per_chunk, verbose)
     67 #   print(inputs)
     68   chunks_list = []
---> 69   for prediction_chunk in chunked_prediction_generator(
     70       predictor_fn=predictor_fn,
     71       rng=rng,
     72       inputs=inputs,
     73       targets_template=targets_template,
     74       forcings=forcings,
     75       num_steps_per_chunk=num_steps_per_chunk,
     76       verbose=verbose):
     77     chunks_list.append(jax.device_get(prediction_chunk))
     78   return xarray.concat(chunks_list, dim="time")

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/rollout.py:165, in chunked_prediction_generator(predictor_fn, rng, inputs, targets_template, forcings, num_steps_per_chunk, verbose)
    163 # Make predictions for the chunk.
    164 rng, this_rng = jax.random.split(rng)
--> 165 predictions = predictor_fn(
    166     rng=this_rng,
    167     inputs=current_inputs,
    168     targets_template=current_targets_template,
    169     forcings=current_forcings)
    171 next_frame = xarray.merge([predictions, current_forcings])
    173 current_inputs = _get_next_inputs(current_inputs, next_frame)

/mnt/d/kt/project/yl/google-graphcast/code/test_graphcast.ipynb 单元格 15 line 6
     64 def drop_state(fn):
---> 65   return lambda **kw: fn(**kw)[0]

    [... skipping hidden 12 frame]

File ~/soft/anaconda3/lib/python3.10/site-packages/haiku/_src/transform.py:456, in transform_with_state.<locals>.apply_fn(params, state, rng, *args, **kwargs)
    454 with base.new_context(params=params, state=state, rng=rng) as ctx:
    455   try:
--> 456     out = f(*args, **kwargs)
    457   except jax.errors.UnexpectedTracerError as e:
    458     raise jax.errors.UnexpectedTracerError(unexpected_tracer_hint) from e

/mnt/d/kt/project/yl/google-graphcast/code/test_graphcast.ipynb 单元格 15 line 3
     27 @hk.transform_with_state
     28 def run_forward(model_config, task_config, inputs, targets_template, forcings):
     29   predictor = construct_wrapped_graphcast(model_config, task_config)
---> 30   return predictor(inputs, targets_template=targets_template, forcings=forcings)

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/autoregressive.py:212, in Predictor.__call__(self, inputs, targets_template, forcings, **kwargs)
    209     one_step_prediction = hk.remat(one_step_prediction)
    211 # Loop (without unroll) with hk states in cell (jax.lax.scan won't do).
--> 212 _, flat_preds = hk.scan(one_step_prediction, inputs, scan_variables)
    214 # The result of scan will have an extra leading axis on all arrays,
    215 # corresponding to the target times in this case. We need to be prepared for
    216 # it when unflattening the arrays back into a Dataset:
    217 scan_result_template = (
    218     target_template.squeeze('time', drop=True)
    219     .expand_dims(time=targets_template.coords['time'], axis=0))

File ~/soft/anaconda3/lib/python3.10/site-packages/haiku/_src/stateful.py:643, in scan(f, init, xs, length, reverse, unroll)
    637 # We know that we don't need to thread params in and out, since for init we
    638 # have already created them (given that above we unroll one step of the scan)
    639 # and for apply we know they are immutable. As such we only need to thread the
    640 # state and rng in and out.
    642 init = (init, internal_state(params=False))
--> 643 (carry, state), ys = jax.lax.scan(
    644     stateful_fun, init, xs, length, reverse, unroll=unroll)
    645 update_internal_state(state)
    647 if running_init_fn:

    [... skipping hidden 9 frame]

File ~/soft/anaconda3/lib/python3.10/site-packages/haiku/_src/stateful.py:626, in scan.<locals>.stateful_fun(carry, x)
    623 with temporary_internal_state(state):
    624   with base.assert_no_new_parameters(), \
    625        base.push_jax_trace_level():
--> 626     carry, out = f(carry, x)
    627   reserve_up_to_full_rng_block()
    628   carry = (carry, internal_state(params=False))

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/autoregressive.py:183, in Predictor.__call__.<locals>.one_step_prediction(inputs, scan_variables)
    181 # Add constant inputs:
    182 all_inputs = xarray.merge([constant_inputs, inputs])
--> 183 predictions: xarray.Dataset = self._predictor(
    184     all_inputs, target_template,
    185     forcings=forcings,
    186     **kwargs)
    188 next_frame = xarray.merge([predictions, forcings])
    189 next_inputs = self._update_inputs(inputs, next_frame)

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/normalization.py:156, in InputsAndResiduals.__call__(self, inputs, targets_template, forcings, **kwargs)
    154 norm_inputs = normalize(inputs, self._scales, self._locations)
    155 norm_forcings = normalize(forcings, self._scales, self._locations)
--> 156 norm_predictions = self._predictor(
    157     norm_inputs, targets_template, forcings=norm_forcings, **kwargs)
    158 return xarray_tree.map_structure(
    159     lambda pred: self._unnormalize_prediction_and_add_input(inputs, pred),
    160     norm_predictions)

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/casting.py:56, in Bfloat16Cast.__call__(self, inputs, targets_template, forcings, **kwargs)
     52   return self._predictor(inputs, targets_template, forcings, **kwargs)
     54 with bfloat16_variable_view():
     55   predictions = self._predictor(
---> 56       *_all_inputs_to_bfloat16(inputs, targets_template, forcings),
     57       **kwargs,)
     59 predictions_dtype = infer_floating_dtype(predictions)
     60 if predictions_dtype != jnp.bfloat16:

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/casting.py:179, in _all_inputs_to_bfloat16(inputs, targets, forcings)
    164 def _all_inputs_to_bfloat16(
    165     inputs: xarray.Dataset,
    166     targets: xarray.Dataset,
   (...)
    177 #   data_vars = {key: value.values if hasattr(value, 'values') else value for key, value in inputs.items()}
    178 #   dataset = xr.Dataset(data_vars)
--> 179     return (inputs.astype(jnp.bfloat16),
    180             jax.tree_map(lambda x: x.astype(jnp.bfloat16), targets),
    181             forcings.astype(jnp.bfloat16))
AttributeError: 'dict' object has no attribute 'astype'

However, running the code with the example data works correctly. The specific error message encountered when forecasting with my processed GFS data. I would appreciate any assistance in resolving this issue

I'm attempting to use processed GFS (Global Forecast System) data for forecasting. The processed data's coordinates, dimensions, and variables are nearly identical to the example data provided. However, when running the Autoregressive rollout code snippet, specifically this segment:
# Autoregressive rollout code snippet
assert model_config.resolution in (0, 360. / eval_inputs.sizes["lon"]), (
  "Model resolution doesn't match the data resolution. You likely want to "
  "re-filter the dataset list, and download the correct data.")

print("Inputs:  ", eval_inputs.dims.mapping)
print("Targets: ", eval_targets.dims.mapping)
print("Forcings:", eval_forcings.dims.mapping)

predictions = rollout.chunked_prediction(
    run_forward_jitted,
    rng=jax.random.PRNGKey(0),
    inputs=eval_inputs,
    targets_template=eval_targets * np.nan,
    forcings=eval_forcings)
predictions

I encounter an error message,The error message is as follows.
File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/rollout.py:69, in chunked_prediction(predictor_fn, rng, inputs, targets_template, forcings, num_steps_per_chunk, verbose)
     67 #   print(inputs)
     68   chunks_list = []
---> 69   for prediction_chunk in chunked_prediction_generator(
     70       predictor_fn=predictor_fn,
     71       rng=rng,
     72       inputs=inputs,
     73       targets_template=targets_template,
     74       forcings=forcings,
     75       num_steps_per_chunk=num_steps_per_chunk,
     76       verbose=verbose):
     77     chunks_list.append(jax.device_get(prediction_chunk))
     78   return xarray.concat(chunks_list, dim="time")

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/rollout.py:165, in chunked_prediction_generator(predictor_fn, rng, inputs, targets_template, forcings, num_steps_per_chunk, verbose)
    163 # Make predictions for the chunk.
    164 rng, this_rng = jax.random.split(rng)
--> 165 predictions = predictor_fn(
    166     rng=this_rng,
    167     inputs=current_inputs,
    168     targets_template=current_targets_template,
    169     forcings=current_forcings)
    171 next_frame = xarray.merge([predictions, current_forcings])
    173 current_inputs = _get_next_inputs(current_inputs, next_frame)

/mnt/d/kt/project/yl/google-graphcast/code/test_graphcast.ipynb 单元格 15 line 6
     64 def drop_state(fn):
---> 65   return lambda **kw: fn(**kw)[0]

    [... skipping hidden 12 frame]

File ~/soft/anaconda3/lib/python3.10/site-packages/haiku/_src/transform.py:456, in transform_with_state.<locals>.apply_fn(params, state, rng, *args, **kwargs)
    454 with base.new_context(params=params, state=state, rng=rng) as ctx:
    455   try:
--> 456     out = f(*args, **kwargs)
    457   except jax.errors.UnexpectedTracerError as e:
    458     raise jax.errors.UnexpectedTracerError(unexpected_tracer_hint) from e

/mnt/d/kt/project/yl/google-graphcast/code/test_graphcast.ipynb 单元格 15 line 3
     27 @hk.transform_with_state
     28 def run_forward(model_config, task_config, inputs, targets_template, forcings):
     29   predictor = construct_wrapped_graphcast(model_config, task_config)
---> 30   return predictor(inputs, targets_template=targets_template, forcings=forcings)

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/autoregressive.py:212, in Predictor.__call__(self, inputs, targets_template, forcings, **kwargs)
    209     one_step_prediction = hk.remat(one_step_prediction)
    211 # Loop (without unroll) with hk states in cell (jax.lax.scan won't do).
--> 212 _, flat_preds = hk.scan(one_step_prediction, inputs, scan_variables)
    214 # The result of scan will have an extra leading axis on all arrays,
    215 # corresponding to the target times in this case. We need to be prepared for
    216 # it when unflattening the arrays back into a Dataset:
    217 scan_result_template = (
    218     target_template.squeeze('time', drop=True)
    219     .expand_dims(time=targets_template.coords['time'], axis=0))

File ~/soft/anaconda3/lib/python3.10/site-packages/haiku/_src/stateful.py:643, in scan(f, init, xs, length, reverse, unroll)
    637 # We know that we don't need to thread params in and out, since for init we
    638 # have already created them (given that above we unroll one step of the scan)
    639 # and for apply we know they are immutable. As such we only need to thread the
    640 # state and rng in and out.
    642 init = (init, internal_state(params=False))
--> 643 (carry, state), ys = jax.lax.scan(
    644     stateful_fun, init, xs, length, reverse, unroll=unroll)
    645 update_internal_state(state)
    647 if running_init_fn:

    [... skipping hidden 9 frame]

File ~/soft/anaconda3/lib/python3.10/site-packages/haiku/_src/stateful.py:626, in scan.<locals>.stateful_fun(carry, x)
    623 with temporary_internal_state(state):
    624   with base.assert_no_new_parameters(), \
    625        base.push_jax_trace_level():
--> 626     carry, out = f(carry, x)
    627   reserve_up_to_full_rng_block()
    628   carry = (carry, internal_state(params=False))

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/autoregressive.py:183, in Predictor.__call__.<locals>.one_step_prediction(inputs, scan_variables)
    181 # Add constant inputs:
    182 all_inputs = xarray.merge([constant_inputs, inputs])
--> 183 predictions: xarray.Dataset = self._predictor(
    184     all_inputs, target_template,
    185     forcings=forcings,
    186     **kwargs)
    188 next_frame = xarray.merge([predictions, forcings])
    189 next_inputs = self._update_inputs(inputs, next_frame)

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/normalization.py:156, in InputsAndResiduals.__call__(self, inputs, targets_template, forcings, **kwargs)
    154 norm_inputs = normalize(inputs, self._scales, self._locations)
    155 norm_forcings = normalize(forcings, self._scales, self._locations)
--> 156 norm_predictions = self._predictor(
    157     norm_inputs, targets_template, forcings=norm_forcings, **kwargs)
    158 return xarray_tree.map_structure(
    159     lambda pred: self._unnormalize_prediction_and_add_input(inputs, pred),
    160     norm_predictions)

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/casting.py:56, in Bfloat16Cast.__call__(self, inputs, targets_template, forcings, **kwargs)
     52   return self._predictor(inputs, targets_template, forcings, **kwargs)
     54 with bfloat16_variable_view():
     55   predictions = self._predictor(
---> 56       *_all_inputs_to_bfloat16(inputs, targets_template, forcings),
     57       **kwargs,)
     59 predictions_dtype = infer_floating_dtype(predictions)
     60 if predictions_dtype != jnp.bfloat16:

File ~/soft/anaconda3/lib/python3.10/site-packages/graphcast/casting.py:179, in _all_inputs_to_bfloat16(inputs, targets, forcings)
    164 def _all_inputs_to_bfloat16(
    165     inputs: xarray.Dataset,
    166     targets: xarray.Dataset,
   (...)
    177 #   data_vars = {key: value.values if hasattr(value, 'values') else value for key, value in inputs.items()}
    178 #   dataset = xr.Dataset(data_vars)
--> 179     return (inputs.astype(jnp.bfloat16),
    180             jax.tree_map(lambda x: x.astype(jnp.bfloat16), targets),
    181             forcings.astype(jnp.bfloat16))
AttributeError: 'dict' object has no attribute 'astype'

However, running the code with the example data works correctly. The specific error message encountered when forecasting with my processed GFS data. I would appreciate any assistance in resolving this issue

Could you share how you are processing your GFS data? I am also trying to initialise with the GFS and am having issues.

您能提供完整的转储吗example_batch？通过复制模型的精确结构，我们在运行自回归部署时没有遇到任何问题。example_batch
Thank you for your response. The issue has been resolved now; it seems that there was an error in handling the input data.


Creating Verbose, User-Friendly Error Messaging

Problem statement
As a Sender who uploads my CSV file, I need to understand the content of error messaging so that I can make adjustments to my file with confidence.
What you need to know
Design
In the Figma File, you will see see a tab called "Error States" it shows the current and future states.

https://www.figma.com/file/ctpYEoOBWGkUnk9cmXzVp2/ReportStream-Submitter-Front-End?node-id=943%3A4273

Acceptance criteria

Given I

Am logged into ReportStream as a user with Sender permissions

When I

Navigate to the new Sender submission page
Choose a file to upload (i.e. browse or drag/drop)

Then I

Can upload my file and the System

Sends the data to ReportStream validator
ReportStream produces one of the following:
1) success message
2) error message
3) error message with additional alerts (warnings that are optional to address)
4) success message with additional alerts (warnings that are optional to address)
After reviewing error/warning messages, I am able to successfully edit my file with confidence
Upload my edited file for submission
Constraint: I am prohibited from uploading a non-CSV file

To do

[ ] Dev: Create a full inventory of error and warning messages in current state
[ ] Design: After full inventory created, develop copy appropriate for non-technical users
[ ] Dev: Map the verbose copy to the current state error messages so that verbose copy will appear in CSV Uploader
[ ] Dev: Test that mapping has occurred correctly and completely

adf

Entered Jim Duff's feedback into spreadsheet: https://docs.google.com/spreadsheets/d/1vrm_TudKG9SntHhm8BgrdOLs_fIIr6KeAzbqnlgNXeg/edit#gid=1323508253
Asked for Tom's clarification on several comments - responses in spreadsheet

Asked Tom if each error had sub-errors like he produced for lines 13 and 14. If so, we have a lot of copy to write.

Tom provided 'sub-errors' to address Jim Duff's feedback. It came to our attention that the other parent errors in the spreadsheet also likely have multiple sub-errors that need to be documented. We brought this to Tom's attention but then Data Robot's contract ended.
There are two pieces up in the air at this point:

We need to get a full inventory of all errors.
Once completed, these errors will need to be mapped in the backend of the system. Tom was originally going to do this and is now gone.

@wwilliams-Agile6 - The inventory of errors to date may be complete but I am NOT sure. In lines 13 and 14, there are 'sub-errors' that are produced. I understand that these are more specific and would be bubbled up to a more generic error. However....

Jim has requested that user's fix any issues with their file themselves rather than contacting the help desk. Therefore, RS would need to bubble up the specific rather than a generic error. Is this possible? Lines 9. 13, 14, 17.

I need you to look into the sub-errors for lines 9 and 17. These are the remaining two in which we currently direct the users to contact the help desk.

@hie1 and @wwilliams-Agile6 I am confused as to what work this ticket involves; typically the "Then" portion of the AC describes the new work that needs to be done but this ticket includes work from all of the other tickets for the uploader, so what work needs to be done as part of this ticket?


develop branch: If an emails body is empty but there is an attachment, then the attachment is included as text

Related: https://github.com/django-helpdesk/django-helpdesk/pull/680

I've been working on this bug for a while without success. It is hard to replicate. My guess is that it might have to do with "application/ms-tnef" maintype's of the message.
But you might actually be right and this might have to do with any messages with attachment, but no body. Testing it right now.

Just tested it and the problem indeed occurs when messages without body and with attachment are imported.
Bug confirmed.

Thanks for testing @johanovic! Could you please add your test email to our unit testing framework so we can make sure we properly handle this situation in the future? Email is notoriously hard to get right; the email processing is probably an older part of the code base and can probably use some clean up to make sure we're using the latest features of python email libraries, etc.

@johanovic Is this happening on master as well, or only develop?

I wish I had a proper test, but I've mostly been testing it step by step.
I have tried to make the changes of https://github.com/django-helpdesk/django-helpdesk/commit/eee82015f787c303acdec85d667a58296093cac2#diff-01c02c5839198cb2c27ec0ef4dc09a16
but sadly they do not solve the issue.

Here is a failing test case: https://github.com/django-helpdesk/django-helpdesk/pull/720

Thanks @timthelion! I just pushed a potential fix. It looks like we were using the last known part of the mime message for processing rather than the message as a whole, so I updated it to use the full message and keep the description empty unless an HTML part is discovered.

I tried this in production and I confirm that it works for emails without a body and without as subject, but with an attachment. Thanks!

Thanks for testing @johanovic! Glad to have production confirmation of the fix :-)


Fixed EscapeMarkdown for \ symbol

You forgot to escape the \ character in the EscapeMarkdown function, which caused me some errors.
Details in this pr from a similar library that also has this disease:
https://github.com/go-telegram-bot-api/telegram-bot-api/pull/604

According to the documentation, the symbol '/' should not be escaped. If you add this character for any reason, you should escape it manually.


Support for SNI and dynamic certificate

Dynamically generate a certificate based on client request using
Server Name Idicator
Sign the new certificate with either a static CA certificate, or
with a newly generated CA
Add config options to specify a path to static CA certificate
*** NOTE ***: This version only works on windows platform

Great Pull Request :+1:
Is there any specific reason this was not merged ?
I also don't understand why it is noted as working only on Windows. I tested it in a Linux multihost setup and it works great (with some minor fixes https://github.com/gaelmuller/flare-fakenet-ng/commit/549e89adf0198cbbfc6a8ef75a79d1d58a97db30)

@gaelmuller This works well on windows systems because we can use certutil to import our own root CA cert into Windows trust store. That will make SSL clients trust everything we sign. However, on Linux, there are a few issues:

With multihost mode: You have to add your own cert into the client cert chain somehow. Current implementation allows you to configure a static cert which can be used to sign other certs as request coming in. This works as intended. Since it requires additional steps, I don't consider it "working out of the box" on linux.

Within singlehost mode: I am not aware of a generic system wide tool to "trust" a signing cert. Also, I can't support most/all distribution, so I also don't consider this feature working out of the box.

This also lacked any fallback for when SNI can't access ssl.SSLContext due to Python version < 2.7.9 (which was still true for supported LTS releases of Ubuntu at the time this PR was considered). So, merging this branch would have broken some use cases of FakeNet on supported distros. The project still needs to migrate to Python3, which might resolve this specific issue.

@htnhan Could you sign Google's CLA agreement? You can find it here: https://cla.developers.google.com/clas
Thanks, Nhan!

@htnhan Could you sign Google's CLA agreement? You can find it here: https://cla.developers.google.com/clas
Thanks, Nhan!

Signed.
Thank you @tinajohnson for merging this :)

🥳


DrillSideways optimizations

Description
This change makes use of advance instead of next where possible and splits out 1st and 2nd phase checking to avoid match confirmation when unnecessary.
Note that I only focused on the doQueryFirstScoring implementation here and didn't modify the other two scoring approaches. "Progress not perfection" and all that (plus, I think we should strongly consider removing these other two implementations, but we'd want to benchmark to be certain).
Unfortunately, luceneutil doesn't have dedicated drill sideways benchmarks, but some benchmarks on our internal software that makes use of drill sideways showed a +2% QPS improvement and no obvious regressions.

Changes LGTM, do we need to add some unit tests?

Changes LGTM, do we need to add some unit tests?

Thanks @zhaih. Let me consider some specific test for this. I know our randomized testing for DrillSideways covers these code paths but maybe some specific, non-random tests would be useful.

@zhaih I reexamined our test coverage and think we're in good shape already actually. We've got good coverage for covering drill-sideways correctness with multiple dimensions, etc. (including random and non-random). We could try to take these further by somehow asserting that advance is being used in favor of nextDoc when appropriate, but I think those tests would be reasonably complex to write and I'm not sure they add tremendous value. I'd rather we spent time building drill-sideways benchmarks that focus on ensuring our performance doesn't regress. But that's just my opinion. Please let me know if you feel differently and we can keep discussing. Thanks!

@zhaih that's a good point and valid concern. I dug into the existing tests and it looks like we have lots of coverage except that the majority of the coverage is using basic, single-phase drill-down dimensions. I'm going to augment our randomized testing to randomly use two-phase drill-downs to broaden coverage. Thanks for the discussion!

@gsmiller Thank you for checking and continuous effort!

@zhaih well, thank you for keeping me honest with testing. I think I've already found an insidious, potential bug with some beefier tests.


Come up with an idea for generic mutable references

Generic Mutable References
Problem
Currently wrappers around objects stored on the contract storage do not provide a way to receive mutable references (&mut T) to those objects.
Rational
The reason for this is that users could do arbitrary changes to the objects without having an automated way to synchronize the storage.
Example
Storage wrappers like storage::Vec and storage::HashMap provide APIs like mutate_with with which it is possible for users to do an efficient in-place mutation of an object owned by the collection.
Goal
This is the tracking issue to come up with a user friendly, efficient and possibly general way to allow for mutable references that keep their referenced objects in sync between memory and storage.
Solutions
Solution 1: Do not provide mutable reference wrappers
This is the naive approach that we currently use.
However, it has the downside that it simply doesn't support what some users would like to have.
Solution 2: ?

This is very valuable feedback!
Having compile time checks would be greatly appreciated we should really try to target having them.
For this to work we need to change how Env in pdsl interoperates with storage allocation.
For example, at the moment Key is seen as raw pointer having no lifetime that could be bound to a certain environment which was a sane default since environments so far were (stateless) wrappers around the SRML provided interfaces. However, if we want to make this change (and I really want to experiment with this) we would need to change that.

Btw.: The flushing system has already been implemented. I am currently trying to resolve some issues around storage allocation and initialization and will then continue with this.

Note that since the flushing system has been successfully implemented and with commit https://github.com/Robbepop/pdsl/commit/c3a942be1cedcb3544362be8a96b9e4ad07e8fa7 even storage::Value now has a DerefMut implementation which allows for very abstract mutable accesses just like in normal Rust code.
So far it seems to work very well.
tldr; I think we can close this issue.


protocol: quickHTTP

A Serverlessly Protocol resembling HTTP/1.1 (1999) which can serialize raw data buffer coming from TCP socket as per RFC2616 standard. It's meant for short-lived microservices which don't require stream or platforms like AWS Lambda which don't support stream.
Name: HTTP/1.1 (No Stream)
Protocol Context:
Under Review..
It shouldn't contain any stream (responsibility of stream chunk collection goes to Platform Adapters)

This is a well-defined :rocket: Feature which needs to be implemented

No need to complicate stuff by having an extra protocol for short-lived microservices which can also limit the ecosystem. This can be assimilated in https://github.com/ServerlesslyStack/Serverlessly/issues/26.


Design for entities and entity logic

Initial commit for dev branch - we are based off the commits done in credesign. The dev branch should be where work in progress changes should be committed to, and breakage should be expected! main is the production branch and should be considered stable. Any changes made to main should be discussed and reviewed!
Developers, in their own forks, will create PRs to merge their changes into the dev branch. From there, we will decide when to merge dev into main.

Resolved / Related Issues

[x] Were these changes approved in an issue or discussion with the project maintainers? In order to prevent extra work, feature requests and changes to the codebase must be approved before the pull request will be reviewed. This prevents extra work for the contributors and maintainers.
Closes #4

Validation
How did you test these changes?

[ ] Did you run the python notebook and test your changes?
[x] Did you implement any design changes to an existing feature?

[ ] Was this change approved?

[x] Are there any dependencies that have yet to be constructed?
Relies on ...
[ ] Are there any other steps that were used to validate these changes?

Unit/Function
Discuss with AI side

Screenshots (optional)
Add screenshots here.

This was addressed in #22


Investigate and implement reliable subscriptions filtering

Subscription filtering is curently disable due to the limitations of the mechanism. We should investigate other frameworks to see how we can apply basic filtering capabilities to subscriptions without sacrificing performance

So what is the state of subscriptions right now? Do they just work for all updates to an entity, all creates to an entity, or all deletes to an entity?

Subscriptions do not have filtering enabled. Every update to dataset will send update to client. We have filtering on the schema it is just to apply it so it will actually perform.
Filtering described on subscription package is not production ready.

@lastmjs Looking for feedback. I'm trying to push some of the structure into subscriptions - adding docs first and collecting requirements.
Would you prefer different subscriptions per CRUD model - CREATE, DELETE, UPDATE
or a single subscription that offers extra fields:
_created: Boolean
_updated: Boolean
_deleted: Boolean

CC @craicoverflow @machi1990

Personally I would prefer separate subscriptions per CRUD model, similar to how it is now: updatedUsers, createdUsers, deletedUsers. This seems to match the mutations more directly, making the context switch simpler, and I imagine would allow the same or similar filters to be used on the subscription as in the mutations.

Let's remove subscription filters from the schema for now since they are not fully implemented. It can be restored from https://github.com/aerogear/graphback/pull/1683 when required.

From my point this will cause couple issues related to offix and client side queries. If we have ability to not break client side API/typings while we can provide this feature under the hood after 0.14.0 with smooths and painless update. If we are worried about users being confused we can put warning for filtering

Sure, I did not really consider how it would affect documentation. I will add it to docs.

Added this to 0.15 milestone. Is that okay?

yep.. I have missed that ! Thank you

Fixed by #1748.  We released 0.15.0-alpha1 containing this feature, with further updates to follow as part of https://github.com/aerogear/graphback/issues/1760


storage: add deadline check in get and batch_get request handler

What is changed and how it works?

Issue Number: Close #15306
What's Changed:

add deadline check in get and batch_get request handler.

Related changes

N/A

Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)

Side effects

Performance regression

Consumes more CPU

Release note
None.

@crazycs520
Unit test cases to cover timeout error for get, batch get and cop tasks are needed too.

Shall we also check it for batch_get_commands?

I already checked batch_get in this pr.

Shall we also check it for batch_get_commands?

I already checked batch_get in this pr.

I meant this https://github.com/tikv/tikv/blob/4fb70fbee527916f7e74552d3e270911c2b8334e/src/storage/mod.rs#L758
I'm also wondering if it's necessary to check deadlines in batch_get when some keys have been read and some are not yet.

I'm also wondering if it's necessary to check deadlines in batch_get when some keys have been read and some are not yet.

Maybe it's necessary because the client will drop the result when deadline exceeds.

Shall we also check it for batch_get_commands?

I already checked batch_get in this pr.

I meant this
https://github.com/tikv/tikv/blob/4fb70fbee527916f7e74552d3e270911c2b8334e/src/storage/mod.rs#L758

Nice catch, already fixed. PTAL

/merge

@crazycs520
Please fix the lint errors
[2023-08-17T12:43:51.580Z] + cd /home/jenkins/tikv-src

[2023-08-17T12:43:51.581Z] + export RUSTFLAGS=-Dwarnings

[2023-08-17T12:43:51.581Z] + RUSTFLAGS=-Dwarnings

[2023-08-17T12:43:51.581Z] + make format

[2023-08-17T12:43:51.840Z] info: component 'rustfmt' for target 'x86_64-unknown-linux-gnu' is up to date

[2023-08-17T12:43:58.404Z] + git diff --quiet

[2023-08-17T12:43:58.404Z] + git diff

[2023-08-17T12:43:58.404Z] diff --git a/tests/Cargo.toml b/tests/Cargo.toml

[2023-08-17T12:43:58.404Z] index d554417d1..79367c006 100644

[2023-08-17T12:43:58.404Z] --- a/tests/Cargo.toml

[2023-08-17T12:43:58.404Z] +++ b/tests/Cargo.toml

[2023-08-17T12:43:58.404Z] @@ -109,10 +109,10 @@ tidb_query_executors = { workspace = true }

[2023-08-17T12:43:58.404Z]  tidb_query_expr = { workspace = true }

[2023-08-17T12:43:58.404Z]  tikv = { workspace = true }

[2023-08-17T12:43:58.404Z]  tikv_util = { workspace = true }

[2023-08-17T12:43:58.404Z] -tracker = { workspace = true }

[2023-08-17T12:43:58.404Z]  time = "0.1"

[2023-08-17T12:43:58.404Z]  tipb = { workspace = true }

[2023-08-17T12:43:58.404Z]  toml = "0.5"

[2023-08-17T12:43:58.404Z] +tracker = { workspace = true }

[2023-08-17T12:43:58.404Z]  txn_types = { workspace = true }

[2023-08-17T12:43:58.404Z]  uuid = { version = "0.8.1", features = ["serde", "v4"] }

[2023-08-17T12:43:58.404Z]

[2023-08-17T12:43:58.404Z] diff --git a/tests/failpoints/cases/test_storage.rs b/tests/failpoints/cases/test_storage.rs

[2023-08-17T12:43:58.404Z] index 5369c36de..57047bef9 100644

[2023-08-17T12:43:58.404Z] --- a/tests/failpoints/cases/test_storage.rs

[2023-08-17T12:43:58.404Z] +++ b/tests/failpoints/cases/test_storage.rs

[2023-08-17T12:43:58.404Z] @@ -1580,9 +1580,10 @@ fn test_deadline_exceeded_on_get_and_batch_get() {

[2023-08-17T12:43:58.404Z]          vec![INVALID_TRACKER_TOKEN; 1],

[2023-08-17T12:43:58.404Z]          consumer.clone(),

[2023-08-17T12:43:58.404Z]          Instant::now(),

[2023-08-17T12:43:58.404Z] -    )).unwrap();

[2023-08-17T12:43:58.404Z] +    ))

[2023-08-17T12:43:58.404Z] +    .unwrap();

[2023-08-17T12:43:58.404Z]      let result = consumer.take_data();

[2023-08-17T12:43:58.404Z] -    assert_eq!(1,result.len());

[2023-08-17T12:43:58.404Z] +    assert_eq!(1, result.len());

[2023-08-17T12:43:58.404Z]      assert!(matches!(

[2023-08-17T12:43:58.404Z]          result[0],

[2023-08-17T12:43:58.404Z]          Err(StorageError(box StorageErrorInner::DeadlineExceeded))

[2023-08-17T12:43:58.404Z] + echo Please make format and run tests before creating a PR

[2023-08-17T12:43:58.404Z] Please make format and run tests before creating a PR

[2023-08-17T12:43:58.404Z] + exit 1

script returned exit code 1

/merge


Running the contributed command: 'highlightwords.addHighlight' failed.

what's the problem?

I don't understand the question, is the title an error message you recieved?  What are the steps to reproduce?

I don't understand the question, is the title an error message you recieved?  What are the steps to reproduce?

I also met this problem when I want to highlight word in a large file.

I also met this problem when I want to highlight word in a large file.

Command 'Highlight Toggle Sidebar' resulted in an error (Running the contributed command: 'highlightwords.toggleSidebar' failed.)
Command 'Highlight Toggle Current' resulted in an error (Running the contributed command: 'highlightwords.addHighlight' failed.)

Also met this problem, while 'Set Highlight Mode' command running fine.
vscode version info:
Version: 1.52.1
Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523
Date: 2020-12-16T16:30:02.420Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 19.6.0

Command 'Highlight Toggle Sidebar' resulted in an error (Running the contributed command: 'highlightwords.toggleSidebar' failed.)
Command 'Highlight Toggle Current' resulted in an error (Running the contributed command: 'highlightwords.addHighlight' failed.)

Also met this problem, while 'Set Highlight Mode' command running fine.
vscode version info:
Version: 1.52.1
Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523
Date: 2020-12-16T16:30:02.420Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 19.6.0

I can not recreate this issue, if someone could please provide the steps to recreate, thanks.

I can not recreate this issue, if someone could please provide the steps to recreate, thanks.

I follow the Microsoft VSCode Extension dev guide, and setup a dev environment with your source code.
I got these Errors while debugging:
node_modules/@types/node/index.d.ts(66,11): error TS2300: Duplicate identifier 'IteratorResult'.
src/tree.ts(51,6): error TS2611: 'tooltip' is defined as a property in class 'TreeItem', but is overridden here in 'HighlightNode' as an accessor.
../../../../usr/local/lib/node_modules/typescript/lib/lib.es2015.iterable.d.ts(41,6): error TS2300: Duplicate identifier 'IteratorResult'.

After doing these changes, i can run the extension well:

Comment the 'tooltip()' function, for skip 'tooltip' funciton error
Add '"skipLibCheck": true' into "compilerOptions" of 'tsconfig.json' file, for fix Duplicate identifier 'IteratorResult' error

I'm not familiar with typescript, hope these info can be helpful, thanks.

I follow the Microsoft VSCode Extension dev guide, and setup a dev environment with your source code.
I got these Errors while debugging:
node_modules/@types/node/index.d.ts(66,11): error TS2300: Duplicate identifier 'IteratorResult'.
src/tree.ts(51,6): error TS2611: 'tooltip' is defined as a property in class 'TreeItem', but is overridden here in 'HighlightNode' as an accessor.
../../../../usr/local/lib/node_modules/typescript/lib/lib.es2015.iterable.d.ts(41,6): error TS2300: Duplicate identifier 'IteratorResult'.

After doing these changes, i can run the extension well:

Comment the 'tooltip()' function, for skip 'tooltip' funciton error
Add '"skipLibCheck": true' into "compilerOptions" of 'tsconfig.json' file, for fix Duplicate identifier 'IteratorResult' error

I'm not familiar with typescript, hope these info can be helpful, thanks.


Add support for belt

Would you elaborate on the title of this issue?

My bad. I meant exception analysis for belt standard library. eg. Belt.Map.getExn

Here's a template adding Belt.Map and Belt.List: https://github.com/reason-association/reanalyze/pull/112
The template includes the various aliases, such as e.g. Belt_Map. In theory these are not documented, but there's nothing stopping people from using them directly.
Other parts of Belt can be added in the same way, though it does not need more tests, those are only there to check that the mechanism works.

Merged.


running flutter upgrade tries calling non existent command (Windows)

I just upgraded flutter via flutter upgrade from the windows cmd.
The result was the following output:
D:\workspace\b-strauss\flutterapp>flutter upgrade
Upgrading Flutter from D:\workspace\_sdks\flutter...
Updating d32dc9254..a80e5c20f
<EMAIL_ADDRESS>     |  Bin 0 -> 564 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1283 bytes
<EMAIL_ADDRESS>=><EMAIL_ADDRESS>|  Bin
 .../app/src/main/res/mipmap-hdpi/ic_launcher.png   |  Bin 0 -> 544 bytes
 .../app/src/main/res/mipmap-mdpi/ic_launcher.png   |  Bin 0 -> 442 bytes
 .../app/src/main/res/mipmap-xhdpi/ic_launcher.png  |  Bin 0 -> 721 bytes
 .../app/src/main/res/mipmap-xxhdpi/ic_launcher.png |  Bin 0 -> 1031 bytes
 .../src/main/res/mipmap-xxxhdpi/ic_launcher.png    |  Bin 0 -> 1443 bytes
 .../flutter_view/assets/flutter-mark-square-64.png |  Bin 0 -> 482 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1025 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1716 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1920 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1283 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1895 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 2665 bytes
<EMAIL_ADDRESS>     |  Bin
<EMAIL_ADDRESS>     |  Bin 0 -> 2665 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 3831 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1888 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 3294 bytes
<EMAIL_ADDRESS>     |  Bin
<EMAIL_ADDRESS> |  Bin 0 -> 3612 bytes
 examples/flutter_view/ios/Runner/ic_add.png        |  Bin 0 -> 86 bytes
<EMAIL_ADDRESS>            |  Bin 3066 -> 4630 bytes
<EMAIL_ADDRESS>            |  Bin 4853 -> 7707 bytes
 .../Assets.xcassets/AppIcon.appiconset/Icon-76.png |  Bin 1826 -> 2763 bytes
<EMAIL_ADDRESS>            |  Bin 3977 -> 6241 bytes
<EMAIL_ADDRESS>          |  Bin 4423 -> 6981 bytes
 .../AppIcon.appiconset/Icon-Notification.png       |  Bin 0 -> 552 bytes
<EMAIL_ADDRESS>  |  Bin 0 -> 1293 bytes
<EMAIL_ADDRESS>  |  Bin 0 -> 2087 bytes
 .../AppIcon.appiconset/Icon-Small-40.png           |  Bin 1196 -> 1293 bytes
<EMAIL_ADDRESS>      |  Bin 1957 -> 2952 bytes
<EMAIL_ADDRESS>      |  Bin 3066 -> 4630 bytes
<EMAIL_ADDRESS>         |  Bin 1361 -> 1976 bytes
<EMAIL_ADDRESS>         |  Bin 2132 -> 3216 bytes
 .../AppIcon.appiconset/icon_128x128.png            |  Bin 3254 -> 0 bytes
<EMAIL_ADDRESS>       |  Bin 7158 -> 0 bytes
 .../AppIcon.appiconset/icon_16x16.png              |  Bin 425 -> 0 bytes
<EMAIL_ADDRESS>         |  Bin 952 -> 0 bytes
 .../AppIcon.appiconset/icon_256x256.png            |  Bin 7158 -> 0 bytes
<EMAIL_ADDRESS>       |  Bin 16085 -> 0 bytes
 .../AppIcon.appiconset/icon_32x32.png              |  Bin 952 -> 0 bytes
<EMAIL_ADDRESS>         |  Bin 1529 -> 0 bytes
 .../AppIcon.appiconset/icon_512x512.png            |  Bin 16085 -> 0 bytes
<EMAIL_ADDRESS>       |  Bin 27033 -> 0 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 564 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1283 bytes
<EMAIL_ADDRESS>     |  Bin 0 -> 1588 bytes
<EMAIL_ADDRESS>     |  Bin 4867 -> 0 bytes
 .../process/000.git.60403.stderr}                  |    0
 .../process/001.git.60404.stderr                   |    0
 .../process/002.defaults.60405.stderr              |    0
 .../process/005.xcode-select.60406.stderr          |    0
 .../process/006.xcodebuild.60407.stderr            |    0
 .../process/007.clang.60408.stdout                 |    0
 .../process/008.adb.60409.stderr                   |    0
 .../process/009.idevice_id.60410.stderr            |    0
 .../process/010.which.60411.stderr                 |    0
 .../process/011.idevice_id.60412.stderr            |    0
 .../process/011.idevice_id.60412.stdout            |    0
 .../process/012.simctl.60413.stderr                |    0
 .../process/013.unzip.60414.stderr                 |    0
 .../process/013.unzip.60414.stdout                 |    0
 .../process/014.defaults.60423.stderr              |    0
 .../process/015.simctl.60424.stderr                |    0
 .../process/015.simctl.60424.stdout                |    0
 .../process/016.tail.60432.stderr                  |    0
 .../process/017.simctl.60433.stderr                |    0
 .../process/018.tail.60435.stderr                  |    0
 .../process/018.tail.60435.stdout                  |    0
 506 files changed, 19726 insertions(+), 4641 deletions(-)

Upgrading engine...
Updating flutter tool...
Downloading package sky_engine...
Downloading windows-x64 tools...
Downloading android-arm-profile/windows-x64 tools...
Downloading android-arm-release/windows-x64 tools...
Downloading android-x86 tools...
Downloading android-x64 tools...
Downloading android-arm tools...
Downloading android-arm-profile tools...
Downloading android-arm-release tools...
Already up-to-date.

Flutter • channel master • https://github.com/flutter/flutter.git
Framework • revision a80e5c20f3 (7 minutes ago) • 2017-03-15 15:22:54
Engine • revision 2937f06a15
Tools • Dart 1.23.0-dev.2.0

Running 'flutter packages upgrade' in flutterapp...

Running flutter doctor...
[√] Flutter (on Windows, channel master)
    • Flutter at D:\workspace\_sdks\flutter
    • Framework revision a80e5c20f3 (7 minutes ago), 2017-03-15 15:22:54
    • Engine revision 2937f06a15
    • Tools Dart version 1.23.0-dev.2.0

[√] Android toolchain - develop for Android devices (Android SDK 25.0.2)
    • Android SDK at C:\Users\Ben\AppData\Local\Android\sdk
    • Platform android-25, build-tools 25.0.2
    • Java(TM) SE Runtime Environment (build 1.8.0_121-b13)

[√] Android Studio (version 2.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Gradle version 3.2

[√] IntelliJ IDEA Ultimate Edition (version 2016.3)
    • Dart plugin version 163.13137
    • Flutter plugin version <IP_ADDRESS>

[√] Connected devices
    • None
Der Befehl "ls_dirscript_path"" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

The last part is german for "command 'ls_dirscript_path' either misspelled or not found". I have no idea what that is, I didn't type it. Everything seems to be working after the upgrade.
@goderbauer

Danke for the report.
I think I can reproduce this locally, although on my system I get a slightly different message:
'The' is not recognized as an internal or external command,
operable program or batch file.

I'll investigate this further. Let me know if this is happening for you again (for the same or other commands).

I know what is happening here. Windows doesn't load the batch script into memory upfront to execute the version in memory. It fetches every instruction from disk line by line: After executing line x, it will go back to disk, and executes line x+1. That works great most of the time. However, when line x modifies the batch script, line x+1 might not be the right line to execute next.
And that's exactly what's happening here: flutter upgrade really executes flutter.bat upgrade. The command updates your flutter checkout to the latest version. If flutter.bat has changed, it will update that file while it is executing. If lines have been added or removed from that file, Windows might execute the wrong x+1 line causing the error you see.
I don't see a good way around this. And given that changes to flutter.bat are going to be extremely rare, we probably don't have to worry about this much.

Also: this problem is not limited to Windows. Shell Scripts on Linux have the same issue when the script is updating itself.

Seems like there isn't much to do here?

I believe this is fixed by https://github.com/flutter/flutter/pull/21085.


Why can i not use topic_prefix: /device/pi ?

Why can i not use topic_prefix: /device/pi ?
The / in the  topic_prefix: /device/pi  dosen't work... it every device/pi :-(

Because of this line: https://github.com/flyte/pi-mqtt-gpio/blob/develop/pi_mqtt_gpio/server.py#L56
I wasn't aware that topics with a leading slash were valid. If that is the case, then I'll change the .strip() to an .rstrip()

That ought to have done the trick. Please don't hesitate to update this ticket if it hasn't.

Thank you very much !


Modules stabilization tracking issue

This issue tracks stabilization of mod statements.

Some can be implemented without breaking backwards compatibility:

Recipes defined in one module can be called from another
Variables defined in one module can be referenced from another
Allow inline modules that are declared and defined in a single file.
Add submodule() and submodule_directory() functions

Some require some thought, to make sure that they can be added without breaking existing justfiles:

Allow using parent justfile settings. If this isn't opt-in, it would lead to the behavior of submodules changing.
Do we want to load dotenv files for submodules? Currently, dotenv files are loaded only for the root justfile. I think this is probably fine, but if we started loading dotenv files for child justfiles, it would be a chnage in behavior.

Found this issue also hoping it was talking about remote includes. We currently use make, and rely heavily on a centralized makefile. It's distributed using an include line that uses a shell command to invoke curl:
include $(shell test -f .common.mk || curl -sSL -o .common.mk http://url/to/common.mk; echo .common.mk)

Works pretty well, but some of the frustrations around make have us always looking for alternatives. Which is how I found just, and now I am evaluating whether it will work for us...

I would also jump in with support for remote modules.
One of the hardest things about working with cloud infrastructure and deploying applications is the need for sharing and reusing.
Wanting to reuse all these great just commands I've written in one repository are not easily transferred to another repository.
I feel like deno has solved this problem (URL imports, versioned, immutable repos), and aim to write all my future cloud related coordination and scripting.
But it doesn't get around not being able to re-use just commands where I really want them.
I think this tool really appeals to those who pipeline software.

I'd love to have something analogous to url imports in dhall for importing modules in just. A simple solution without caching or checksums would be enough for my needs. I'd likely use commit hashs/tags for versioning and whether or not a 50ms http request is cached or not doesn't matter to me.
Fetching from private repos is probably a more important feature and requires some though in the general case, for github nothing special is required as one can just put the personal access token in the url.
Importing from public repo:
import https://raw.githubusercontent.com/casey/just/v0.9.4/examples/keybase.just

Importing from private repo
import<EMAIL_ADDRESS>
where TOKEN is either supplied via the command line or via environment variable.

Thanks a lot for the new module system in v1.19.0 🔥
It'd be amazing to have the following feature (you already envisioned) in subsequent releases:

Allow specifying the path to the module source file, so a module justfile can be anywhere.

This is since I'd like to have one commands/ folder where I put in all my submodule .justfiles, e.g. docker.justfile, test.justfile etc. and have one project root .justfile. Having to create a new folder for every of them only clutters  the workspace. I feel like how you have your project structured in folders does not necessarily correlate to how you want your commands to be modularized.
Here is the way I achieved this behavior so far.

@Splines It is done! Added in #1786. You can now give a path by putting a string after the module name:
mod docker 'commands/docker.justfile'

It should make it into the next release.

@casey Wow, that was quick! Very greateful ❤

Thank you @casey for hanging in there on the import and module features!

Hey @casey, thank you so much for just and for this update, quick question, do you plan, or maybe it's already possible and i've missed it to include modules from git repos, like @stereobutter initially mentioned, i would love that.

@WladyX Do you mean including modules from a local git repo, or a remote git repo?

Yes, exactly.

@WladyX Sorry, to be clear, do you want to load modules from a file in a local git repo, e.g., one that's on the same computer that just is running on, or a remote repo, like one on GitHub?

no worries Casey!
I would like to be able to import a module from a private gitlab instance.
My master plan is to a have repo with just modules and import the modules i want to any repo/folder i need.
eg:

repo1 (aka just-modules) with modules foo1, foo2
repo2 (aka myrepo) with bar that imports foo1 from repo1.mydomain.com

ideally i want to be able to make templates and by using variables to adapt them to each repo.
so when i update a template(module) i update it for every repo using it.
does that make sense?

I created #1799 to discuss remote imports.

I'm finding it to be off putting that the functions justfile() and justfile_directory() change behavior when in a modules file.  I would expect that they always return the current file. Is this what module() and module_directory() are going to be used for?

Should an alias in a parent accept the names of recipes in modules? I am not sure if this might work in a future version of just, or whether it is not expected to be supported.
I am currently doing this in a parent justfile to enable top-level aliases to delegate to recipes in modules:
mod pre-commit

# Run all checks
lint:
    @just pre-commit::check

Ideally, I would like to do something like this:
alias lint := pre-commit::check

Love the idea of this feature for better separation (and no naming clashes like import).
I found one potential issue (maybe I'm just using it wrong :sweat_smile:), which I expected to work:
I've:
| .justfiles/fmt.just
| .justfiles/msrv.just
justfile

With content of .justfiles/clippy.just:
fmt: yare tests-integration tests-ui

yare:
    cargo fmt --all

tests-integration:
    cargo fmt --manifest-path {{justfile_directory()}}/yare-tests-integration/Cargo.toml

tests-ui:
    cargo fmt --manifest-path {{justfile_directory()}}/yare-tests-ui/Cargo.toml

and .justfiles/msrv.just:
msrv: yare yare-macro

yare: install-cargo-msrv
    cargo msrv verify --manifest-path  {{justfile_directory()}}/Cargo.toml

yare-macro: install-cargo-msrv
    cargo msrv verify --manifest-path  {{justfile_directory()}}/yare-macro/Cargo.toml

install-cargo-msrv:
    cargo install cargo-msrv --version 0.16.0-beta.20

and finally justfile:
mod fmt '.justfiles/fmt.just'
mod msrv '.justfiles/msrv.just'

before-push: fmt::fmt msrv::msrv

Potential issue A
I wanted to have before-push depend on the first command in '.justfiles/fmt.just' and the first command in '.justfiles/msrv.just', however, when either defining before-push: fmt::fmt msrv::msrv (path syntax) or before-push: fmt msrv (first command in module) I get:
just --unstable
error: Expected '&&', comment, end of file, end of line, identifier, or '(', but found ':'
 ——▶ justfile:6:17
  │
6 │ before-push: fmt::fmt msrv::msrv
  │                 ^

Then, I figured, I would just call it instead of defining it as a dependency:
mod fmt '.justfiles/fmt.just'
mod msrv '.justfiles/msrv.just'

before-push:
  fmt::fmt

Result:
 just --unstable
fmt::fmt
sh: 1: fmt::fmt: not found
error: Recipe `before-push` failed on line 7 with exit code 127

Hmmm, maybe then:
mod fmt '.justfiles/fmt.just'
mod msrv '.justfiles/msrv.just'

before-push:
  fmt

Result:
In this case, just doesn't return an error, but hangs indefinitely:

In the end I found this did work though:
mod fmt '.justfiles/fmt.just'
mod msrv '.justfiles/msrv.just'

before-push:
    just --unstable fmt
    just --unstable msrv

$ just --version
just 1.25.0

Recipes are external commands, so you likely have a binary called fmt in your PATH that waits for input on stdin, hence the hanging.
The way you've settled on (calling just <recipe>) is the only way I know of to call recipes across module boundaries.
I expect that eventually it will be possible to call module recipes as dependencies like you tried to.

Recipes run external commands, so you likely have a binary called fmt in your PATH that waits for input on stdin, hence the hanging.

Ah of course! Too much tunnel vision I imagine, after I couldn't get the previous solutions to work.

The way you've settled on (calling just <recipe>) is the only way I know of to call recipes across module boundaries.
I expect that eventually it will be possible to call module recipes as dependencies like you tried to.

I think that would be incredibly useful.

@casey
From https://github.com/casey/just/issues/1202#issuecomment-2131169830:

I want to give a final call for testing and feedback for the modules feature. All bugs I know of have been fixed

Would you consider #1866 to be a bug related to modules?

@crdx It's definitely a bug related to modules, although it can be fixed in a backwards-compatible way, so it isn't a stabilization blocker.

On my end, while impressed with the subcommand features implemented with modules and submodules, I am not really satisfied with the listing of the commands in modules and submodules.
I find it too verbose and too nested.
What I would expect, as it fits my taste, would be short lines for subcommands, and not showing the subcommands. So, I'd suggest, as an additional feature, a directive [no-unfold] or [fold-listing] that would only give the command given access to subcommands, but not the nested subcommands.
This way, I have the freedom to implement a help command and would allow to invoke the listing of subcommands of a command and only that.
Working in small tmuxed fractions of screen over ssh, having a long, nested, list of commands, subcommands of commands, subcommands of subcommands of commands... not that good an idea.

I concur @gl-yziquel — here's a side-by-side comparison of the old behavior and one of the old-style way we have done since 2019 https://github.com/casey/just/issues/208#issuecomment-453529888 and have since made way bigger:

And now I can't list the admin tasks with just --unstable admin -l or any such combination — I don't know if I'm doing something wrong or is it impossible with modules to first only show the top-level modules and then dive deeper like before.
The way we've made our just scripts discoverable and how we've taught that here is that we've told developers to first run just — then take a guess which "namespace" the thing you probably want belongs to, e.g. "start", and then run just start — and then pick the command which one thinks is correct. Here's an example of how one could figure out how to start a device simulator of any kind:

just
Oh, it might be "devices" — let's try it: just devices
Ok cool, I'm interested in IE11 testing — just devices ie11
Ok now I know what to call

Again, I don't know if this behavior is reproducible with modules or not. But this is a crucial aspect for us to keep the scripts both maintainable and discoverable at the same time.

Also, spotted that using just --list --list-prefix "some prefix" with the modules prints out the "some prefix" list prefix double for modules. That looks like a bug — but I've never tested out list-prefix earlier. (I'm using just version 1.27.0 in all my screenshots here)

@BCNelson There are a bunch of new functions for getting paths. justfile() and justfile_directory() always return the path to the root justfile. source_file() and source_directory() return the path to the current file, regardless of whether it's a module or a import. module_file() and module_directory() will return the path to the current module. So inside an import, source_file() will return the path to the import file, and module_file() will return the path to the module that contains it. Note that module_file() and module_directory() are still unreleased.

@gl-yziquel and @valscion Thanks so much for the feedback! I'm not a heavy user of modules (in fact, I don't think I've ever actually used one in my own justfiles 😅) so feedback is very helpful.
To sum up:

the default display of submodule recipes is too verbose and nested
just --list should accept a submodule path, so you can do just --list SUBMODULE and it should behave as if you were doing just --justfile SUBMODULE --list
it should be possible to iteratively descend into a tree of modules with --list, so you can do just --list, see stuff, then do just --list SUBMODULE to see things in that submodule, etc

--list-prefix is working as intended. --list-prefix, by default, is four spaces, and you can use --list-prefix to change indentation.
--list output is human readable, not machine readable, so it isn't subject to backwards compatible guarantees, i.e., we can stabilize modules, but still change --list output in the future. That being said, I'd like to get it into a reasonable state before stabilizing modules.

I opened #2107 to track improving --list output with modules.

Something else I noticed, is that using the interactive chooser, e.g. @just --choose --unstable doesn't list anything from the modules (I expected it to).

I figured, if supplied with the module name like @just --choose --unstable dav1d it would show a list from the module, similar to how running just --unstable dav1d would execute the default script in the module dav1d.

@foresterre I wasn't actually able to reproduce this, but I opened #2110 to track it. If you could provide a repro there that would be great.

I just merged #2107, so --list no longer displays recipes in submodules, but instead displays MODULE ....
As far as I know, there are no current blockers to stabilization. There are a lot of features that I want to add later, but they can be added in a backwards compatible way, so they don't block stabilization.
My current plan is to stabilize modules on July 1st, so testing and feedback are much appreciated!

I am thinking about creating a reusable justfile that can work as a standalone file but can also be copied and imported where needed. For example, environment variables in each justfile represent a standalone application (created with a template) in subdirectories (like a monorepo). The main justfile at the root should be able to build all of them, but independently they should work as standalone applications. However, the root justfile should be able to modify the variables (it should also work for recipes) before building (e.g., to give them different names).
Imagine these justfiles created with a template (which anyone can provide with their frameworks, etc.) and placed in different subdirectories within a monorepo:
# app1/justfile
export APP_NAME := "app-name"

build:
    echo building $APP_NAME

# app2/justfile
export APP_NAME  := "app-name"

build:
    echo building $APP_NAME

# justfile
import 'app1/justfile' with {prefix_vars('app1'}
import 'app2/justfile' with {prefix_vars('app2'}

export app1_APP_NAME = "app-name-1"
export app2_APP_NAME = "app-name-2"

build:
    echo "Building...."
    just app1_build
    just app2_build

$ just build
Building...
building app-name-1
building app-name-2

Maybe related https://github.com/casey/just/issues/2215

@valscion Thanks for the detailed write up! I'm really glad to hear that modules are mostly working for your use-case, especially since it was your comment way back in 2019 which got me thinking about modules in the first place.
just --show should definitely work with modules, since there's really no reason not to. Compilation errors currently contain a path to the module in which they occurred, but non-complation errors do not. This is only because compilation errors always contain a token, which contains a path to the source file in which the error occurs, which makes printing a path to the source file easy. Non-compilation errors should definitely have a path to the justfile, since otherwise you're right it pretty confusing.
I added both of these to the list of future changes to make in the top comment, since both of them can be implemented in a backwards compatible fashion. When modules are stabilized and this issue is closed, I'll make dedicated issues for all of them.

Is there any reason why / is not allowed in a mod name? I'm looking for a way to distinguish a mod so that I can quickly know it's coming from a specific directory. Using - or _ is not a solution imho.
mod 'dir1/dir2/dir3' './dir1/dir2/dir3/justfile'

build:
    dir1/dir2/dir3::build

Module names are identifiers, which are of the form [a-zA-Z_][a-zA-Z9-9-]*. This is intentionally restrictive. For example, if / were allowed in identifiers, then a/b couldn't be used as the path joining operator, since it would be parsed as an identifier.

Perhaps a "mod mymod from myfile" syntax would be nice. I kind of stumbled in the same situation as I'd like some of my module where deployed in some /usr/share/just hierarchy.

@gl-yziquel. Can you do mod mymod "/usr/share/just/some/module.just"?

@gl-yziquel. Can you do mod mymod "/usr/share/just/some/module.just"?

I'll keep that in mind...

@bukowa @casey Perhaps a "mod mymod from myfile" syntax would be nice. I kind of stumbled in the same situation as I'd like some of my module where deployed in some /usr/share/just hierarchy.

One thing I came across is importing a module with a circular dependency, where examples uses main, and main uses examples. My solution is to split this module into two files, for example:
root/build.just:
build: deps
    ...

deps:
    ...

root/justfile
import './build.just'
mod examples './examples/justfile'

test: build
    just examples::run

root/examples/justfile:

mod root '../build.just'

deps:
    just root::build

run: deps
    ...

Trying to figure out how to reexport var from other module I wonder if is there a better way?
X:
export INTERNAL_VAR := "..."

INTERNAL_VAR:
    echo "{{INTERNAL_VAR}}"

Y:
mod X '../X/justfile'

export SOME_ENV := `just --justfile ../X/justfile INTERNAL_VAR`

Attribute could be a solution here to not repeat recipe hmm
[echo]
export INTERNAL_VAR := "..."

As of just 1.31.0, just released, modules have been stabilized!
There aren't any major changes to modules 1.31, the only minor one is that a mod statement with a path may now point to a directory:
mod foo 'some/directory'

In which case some/directory is searched for a file called justfile, .justfile, or mod.just, so you don't need to write the filename explicitly.
There are still a lot of things missing, but they can be added in a backwards compatible fashion. I'm going to create issues for all the items in the top post here, and make a module improvements tracking issue.
Thanks to everyone for bearing with me, thanks to the contributors and commenters, and thanks to @neunenak who did the initial work with !include that got things going.

The module stabilization tracking issue is dead, long live the module improvement tracking issue!


Bar to promote open source project and also fixing footer

a footer was not 100% because of it used container xxl and not container fluid which it makes look bad but now I already made changes so it is always 100% width whatever the screen size is, and also i already make some bar on top of navbar so we can promote this repo, thanks.

thanks, mas. I will keep an eye on this project because I get used to Laravel and I really comfortable to develop using it


Add argument --no-use-tcmalloc

This PR allows the Linux SDK to be built without tcmalloc. It would partially address https://github.com/dart-lang/sdk/issues/40906
By building Linux SDK without tcmalloc, the SDK would run on musl-libc with gcompat. It’s not perfect but still much better than no support at all.

Just learned that this can be done with export DART_GN_ARGS="dart_use_tcmalloc=false", so it is not necessary to add an other option.


feat: Add tool for patching old logfiles

Motivation
We want to use old logfiles in our new logfile format

fyi. @XianzheMa

Thank you so much! Let me check out your branch to test first

I see. The old rho pipeline configs are not useful anyway. Thanks!
Maximilian Böther @.***> schrieb am Mo. 8. Juli 2024 um
20:27:

@.**** commented on this pull request.
In analytics/tools/patch_eval_format/main.py
https://github.com/eth-easl/modyn/pull/568#discussion_r1669093103:

+from analytics.app.data.load import list_pipelines
+from analytics.tools.patch_eval_format.patch_eval import patch_logfile
+
+
+def main(

logs_directory: Annotated[Path, typer.Argument(help="Path to read the pipelines in from")],
patched_log_dir: Annotated[Path, typer.Argument(help="Path to output the patched pipelines to")],
+) -> None:

find the groups of equivalent pipelines via the .name file

pipelines = list_pipelines(logs_directory)

for pipeline_id, (pipe_name, pipeline_path) in pipelines.items():

   logfile = logs_directory / pipeline_path / "pipeline.log"

   if "rho_loss" in pipe_name:

The problem is that the rho loss config changed since running the old
pipelines. Hence we cannot parse the old files by just porting the
evaluation stuff
—
Reply to this email directly, view it on GitHub
https://github.com/eth-easl/modyn/pull/568#discussion_r1669093103, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AX7NDVMXON6GNHQZNONOFY3ZLLKXVAVCNFSM6AAAAABKRHNKDWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCNRUGA3DQMRQGE
.
You are receiving this because you were mentioned.Message ID:
@.***>

I think we can merge this. The dataset id discrepancy and the eval request thingy are irrelevant to this PR

@XianzheMa @MaxiBoether can one of you approve this PR?

sorry I forgot this


add a licence

If it is a free software, I would like to contribute.

Please feel free to contribute, I'll add the license ASAP

Well, to be honest, we already started! ;-) we made some hacks to make it work on angular2 and add some events from ionic's search bar.
https://github.com/virgilio/ionic2-autocomplete/tree/angular2
What about keep a version angular2/ionic2 compatible? I think the code on master branch works fine on angular2, but we have changed package.json to install on ionic2 app.

@brunosmartin what the difference between this and yours version? 👍

Added license


Expose which webhook endpoint is failing in AlertmanagerClusterFailedToSendAlerts

We have multiple webhook configurations for a receiver. If one of these webhook endpoint is failing so we can't send alerts to them, the AlertmanagerClusterFailedToSendAlerts triggers, but it doesn't say which webhook is affected.
It would be great if the metric "alertmanager_notifications_failed_total" would include a label which is the endpoint url itself, so I could include it in the message.
What did you do?
Configured multiple webhooks and tried to discover which one is failing exactly.
What did you expect to see?
The exact webhook endpoint url, which is failing to send alerts to.
What did you see instead? Under which circumstances?
Only the ingeration label is exposed, which has the value "webhook".

Alertmanager version:

0.19.0

Prometheus version:

2.13.1

I think the problem with this is that alertmanager_notifications_failed_total is increased doesn't matter the notification type, e.g. webhooks, slack notifications, emails, etc.
Since webhook URL is only used at webhook notifications, it doesn't make sense to add this label to alertmanager_notifications_failed_total.
It would if each notification type had its own counter 🤔

I'd be really interested in having the alertmanager_notifications_failed_total metric have a label for the specific notifier that failed.
In an organization with a centralized alertmanager config, you might have hundreds of people getting a slack/webhook at different places. With the metric as it is now, you can only get information that one of them is failing, this can happen anytime someone renames a slack channel, or their webhook app goes down. But it's hard to tell if it's a big outage or a small outage and you've got to go and peep logs to find out who it is.
In the normal case, it's a single receiver causing the issue, so it'd be really awesome to be able to have an alert that had that info in it so that it could be silenced for a few days while someone fixes their webhook receiving app.
There's obviously a cardinality concern, but I think in this case it's worthwhile.

How can we push this forward?

In terms of current demand, alertmanager_notifications_failed_total metric need add receiver and url two tags

While I get how it would help identifying the scope of the alert (one receiver vs. all failing), I'd be worried to add the receiver label by default because of cardinality concerns. But It might be possible to have it behind a feature flag?
cc @roidelapluie for inputs.

While I get how it would help identifying the scope of the alert (one receiver vs. all failing), I'd be worried to add the receiver label by default because of cardinality concerns. But It might be possible to have it behind a feature flag? cc @roidelapluie for inputs.

Can you elaborate a little bit more on the cardinality concerns? I've never seen more than a dozen or so receivers in total in an alertmanager config. I can only see a problem if there is a myriad of receivers, but I cannot imagine a sensible configuration that uses that many receivers.

Moreover, if you have only one receiver this FR will not change your behavior at all. At the same time - when you have a myriad of receivers you definitely want to know which one exactly is failing and not have 1 alert in general, as integration=webhook|smtp|slack is definitely not enough when you have a bunch of webhooks and smtp routes. The only option to get details now is to open alertmanager logs and check where failures are happening. I would add a breaking change notice in release notes that with an update this metric will be per route and if you need same behaviour just sum() alerts, not think a separate flag really needed, it will be many commits to the helm to get this flag there as well.


provide schemas as module

This exports the protocol schemas as an NPM
module so they can be imported in nRF Asset Tracker
projects that need to send or retrieve messages.

Changes to content are minimal and exact replication of already approved content.

There are still links to the old schemas in the docs.

@GilDev is this a problem for you?

Not directly, it was just for information as some of those links point to 404 errors (for example this one: https://github.com/NordicSemiconductor/asset-tracker-cloud-docs/blob/saga/docs/cloud-protocol/state.reported.schema.json). They could also maybe be outdated at some point compared to the new schemas?

Ah, yes it seems some document does not use a permalink. Can you share which page you found that link on? Thanks!

Sure! The “JSON schema file” hyperlink of the “AWS IoT Device Shadow service and MQTT topics” page contains dead links in the $id and $ref parameters.
These seems to be the only ones I found on this page.

Ah, yes, right! We have that fixed in newer releases, but will fix it for the older branches! Thanks again!


Side Panel Bug

Error:Uncaught NotFoundError: Failed to execute 'removeChild' on 'Node': The node to be removed is not a child of this node.
Using the sample code, trying to navigate between 2 pages using side panel within a single html file.
somehow i am unable to navigate back using the side panel after navigating from the first.

        <button class="btn pull-right icon icon-close show-for-phone-only" data-side-panel-close="true"></button>

Side Panel

Home

Page Two

Settings

Login

Hello,
Thank you for reporting this issue.
I just fixed it!