using jupyterlab components

Hi,
Can i use anywidget to interact with jupyter lab components?
Ive been trying the following as my starting point, but my hunch is this isn't possible or the wrong way to go about it.
(3.6.1 is the notebook version i'm currently using)
import anywidget
class ExampleWidget(anywidget.AnyWidget):
    # anywidget, required #
    _esm = """
    import { JupyterFrontEnd, JupyterFrontEndPlugin } from "https://esm.sh/<EMAIL_ADDRESS>
    """
ExampleWidget()

i get this error
Failed to create view for 'AnyView' from module 'anywidget' with model 'AnyModel' from module 'anywidget'
SyntaxError: The requested module<EMAIL_ADDRESS>does not provide an export named 'webcrypto'

It would be great to have a simple example of using anywidget to interact with Jupyter itself. I think that is something others will find useful.
Thanks.

Hi there. Yes, you just need to add the ?bundle query param since @jupyterlab/application isn't properly packaged for npm as ESM.
import anywidget
class ExampleWidget(anywidget.AnyWidget):
    # anywidget, required #
    _esm = """
    import * as jlab from "https://esm.sh/@jupyterlab/application@3.6.1?bundle";
    export function render() {
      console.log(jlab);
    }
    """
ExampleWidget()

I'd like to keep the issues in this repo related to anywidget and not debugging JS modules that aren't related to the project if possible!

Thanks for the help.
Sorry that the question wasn't  closely enough related to the project. I did hesitate to ask the question but ultimately I thought there wasn't anywhere else better to ask about how to interact with jupyter from anywidget.
Anywidget has made it very easy for a jupyter user to experiment in making widgets and many could be python not js users so i think it likely there will be quite a few issues posted that are only peripherally related to anywidget.
If you are looking for ideas for further examples to include in the documentation I think one that works with the jupyter interface would be interesting.
Thanks again.

Sorry that the question wasn't closely enough related to the project. I did hesitate to ask the question but ultimately I thought there wasn't anywhere else better to ask about how to interact with jupyter from anywidget.

No need to apologize at all! I'm sorry if I was short in my response. This thread (and others) led me to open Discussions for such queries in the future. Thanks for giving anywidget a try.

Anywidget has made it very easy for a jupyter user to experiment in making widgets and many could be python not js users so i think it likely there will be quite a few issues posted that are only peripherally related to anywidget.

Totally agree!

If you are looking for ideas for further examples to include in the documentation I think one that works with the jupyter interface would be interesting.

I've never tried this type of integration. Do you have any examples you could share?


[FLINK-16450][hive] Integrate parquet columnar row reader to hive

What is the purpose of the change
#10922 has introduce ParquetColumnarRowSplitReader, we should enable it in hive.
Brief change log

Modify hive pom to shade parquet
Introduce HiveVectorizedParquetSplitReader

Verifying this change
TableEnvHiveConnectorTest modify cases.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@JingsongLi Thanks for the PR. LGTM overall. Just one minor comment.
And since this changes how flink-connector-hive is packaged, have you run some e2e tests to verify we don't have any conflicts?

Hi @lirui-apache , yes, I have ran all profiles test. And verified some versions in cluster.

Hi, @JingsongLi, I am confused about the benefits of parquet vectorized readering.
I checked spark's vector reading slidess, and find that there are three benefits: avoid virtual function dispatch, better cache locality, avoid unnecessary copy/conversion.
So what is the benefits in our flink parquet vectorized readering? BTW, is there any benchmark?

@luoyuxia Could help me to answer this question? I am quite confued about that.

@herunkang2018 What do you mean by parquet vectorized readering? I see the pr is Integrate parquet columnar row reader. Is your question for this particular pr or others?

@luoyuxia IMO columnar row reader is the same as vectorized readering.

@herunkang2018 Sorry for that.  I mistake it's for introduce HiveMapredSplitReader. I think flink parquet vectorized readering will also share the benefits. But for the detail and benchmark, may be the author can provide much information.


feat(bin/ofs): init cloudfilter

Initialize cloudfilter.

The structure of ofs are changed, I will open a new PR before.

The structure of ofs are changed, I will open a new PR later.

Thanks a lot!


How do I ... am bug am bot

Please create a discussion in https://github.com/microsoft/vcpkg/discussions/new instead.

Please do NOT use vcpkg as your test site, make a test repo to do that.


Fix #773 --no-prompt project directory.

Problem:
Providing empty string in vscode input results with using "--no-prompt" as project directory.
My solution:
I have added empty string check and error message for project directory.
Alternatives:

Changing Forge behavior as atm Forge needs the project directory - if you don't supply it in args it will prompt for it (even with --no-prompt switch).
Default, nonempty, project directory?

I don’t think it’s good fix - this feature used to work with passing just empty string as dir.
I think problem is quotePath dir part - probably it’s returning empty string when dir is empty string and what we need to pass in such case is ””

I have changed the fix.

Thanks!


Layer group toggle to toggle all on

The layer group toggle turns any layer with display true off.
With all layers display: false the toggle is disabled.
The desired behavior is turn all layers on when the group toggle is switched on.

The initial idea for group toggle was to prevent user from requesting all grouped layers at once.
Are you sure we want to pull loads of clusters in one go?

Garbage in; Garbage out. There is nothing we can do against bad configurations. One cluster layer alone may break requests when a gazillion cluster are requested.
Being able to put a hundred cluster layers to be toggled simultaneously inside a single group isn't great but should not be forbidden. Performance can not happen by magic by requires knowledge of the inner workings and creating a set of layers which meet the requirements to turn a specific set of location data into information.
WIth a server, even worse a shared server it is not great if one configuration drags over processes down like a person drowning. In a serverless environment these processes stand alone and we / I don't care if process X dies because of an unreasonable amount of data requested.
Besides their are now fail saves such as request timeouts which will prevent crazy complex queries to block the data source layer and lock additional request pools until the end of times.

Done.


Scatter: issue related to length of connecting lines

Dear ScottPlot Developers,
I've encountered an issue with the scatter plot line lengths after updating to ScottPlot version 5.0.28. In the previous version of ScottPlot, the plotted lines between data points were displayed differently, and the current version shows an extended line length that I would like to adjust.
Im searching now for 3 Hours and i cant help myself...
Thank you!

Hi @Alfred-GitHub1, thanks for reporting this! I'm will mark this as high priority and take a closer look soon.

Can you provide a code sample to help us reproduce the issue?

A new package was released this morning (5.0.29) so confirm the issue is present in the latest version

Thanks!

I am encountering a persistent issue where only four rows of DateTime ticks are displayed, despite multiple attempts to rectify this using various code snippets from the ScottPlot Cookbook. It appears that the modification methods applied are not yielding the expected increase in the number of tick rows.
For your reference, here is the segment of my code responsible for generating the tick marks:
               List<DateTime> timestamps = new List<DateTime>();
               List<long> values = new List<long>();
               using (var reader = await commandDb.ExecuteReaderAsync()) {
                   while (await reader.ReadAsync()) {
                       timestamps.Add(reader.GetDateTime(2));
                       values.Add(reader.GetInt64(1));
                   }
               }
               if (timestamps.Count == 0) {
                   await command.FollowupAsync("No data found for the specified item.");
                   return;
               }
               long mostRecentValue = values.LastOrDefault();
               Plot myPlot = new Plot();
               myPlot.Axes.Title.Label.Text = $"{chosenCategory}: {chosenId}. Current Value: {FormatNumber(mostRecentValue)}";
               myPlot.Axes.Title.Label.FontSize = 22f;

               var xs = timestamps.ToArray();
               var ys = values.Select(v => (double)v).ToArray();
               var myScatter = myPlot.Add.Scatter(xs,ys);
               ScottPlot.Colormaps.Turbo colormap = new();
               ScottPlot.Color color = Generate.RandomColor(colormap);
               myScatter.Color = color;
               myScatter.LineWidth = 3;
               myScatter.MarkerSize = 10;
               myScatter.Label = chosenId;
               myPlot.Axes.DateTimeTicksBottom();
               myPlot.ShowLegend();
               myPlot.Axes.Color(ScottPlot.Color.FromHex("#d7d7d7"));
               myPlot.Grid.MajorLineColor = ScottPlot.Color.FromHex("#404040");
               myPlot.FigureBackground.Color = ScottPlot.Color.FromHex("#181818");
               myPlot.DataBackground.Color = ScottPlot.Color.FromHex("#1f1f1f");
               myPlot.Legend.BackgroundFill.Color = ScottPlot.Color.FromHex("#404040");
               myPlot.Legend.FontColor = ScottPlot.Color.FromHex("#d7d7d7");
               myPlot.Legend.OutlineStyle.Color = ScottPlot.Color.FromHex("#d7d7d7");
               myPlot.Legend.Orientation = Orientation.Horizontal;

               Thank you

@Alfred-GitHub1, thanks for a follow-up! Can you paste a screenshot that shows the undesired behavior? It would be helpful if you can show the correct behavior from the previous version too. Thanks!

I adapted your code to be a standalone console application. Can you clarify what the issue is?
DateTime start = new DateTime(2024, 1, 1);
TimeSpan span = TimeSpan.FromDays(1);
List<DateTime> timestamps = ScottPlot.Generate.DateTime.Consecutive(100, start, span).ToList();
List<long> values = ScottPlot.Generate.RandomWalk(100).Select(x => (long)x).ToList();

ScottPlot.Plot myPlot = new();

myPlot.Add.Palette = new ScottPlot.Palettes.Nord();

var myScatter = myPlot.Add.Scatter(timestamps, values);
myScatter.LineWidth = 3;
myScatter.MarkerSize = 10;

myPlot.Axes.DateTimeTicksBottom();
myPlot.Axes.Color(ScottPlot.Color.FromHex("#d7d7d7"));
myPlot.Grid.MajorLineColor = ScottPlot.Color.FromHex("#404040");
myPlot.FigureBackground.Color = ScottPlot.Color.FromHex("#181818");
myPlot.DataBackground.Color = ScottPlot.Color.FromHex("#1f1f1f");

myPlot.SavePng("test.png", 600, 400).LaunchFile();

ScottPlot 5.0.27

ScottPlot 5.0.29

This may be related, but I just fixed a bug https://github.com/ScottPlot/ScottPlot/issues/3736 and will release 5.0.30 within the hour which changes how tick labels are measured. I'm still unsure what your specific issue was, but there's a change this new package will fix it 👍


Make the configure step more reliable

It appears that in cross-compile environments the location of the
"current" directory cannot be assumed. This fix makes it explicit.

merged! thanks!

merged! thanks!


ranking

Let's add ranking calculation

@rultor release, tag is 0.20

@rultor release, tag is 0.20

@yegor256 OK, I will release it now. Please check the progress here

@rultor release, tag is 0.20

@yegor256 Done! FYI, the full log is here (took me 21min)

Oops! There is no order for job gh:yegor256/jpeek#79.

Oops! Job gh:yegor256/jpeek#79 is not in scope.


[WIP] Implement order list and parts of order details

⚠ Work in progress!
Fixes #2639
Re #2587
TODO

[ ] phone numbers returned as single string while input expects prefix + number #2765
[x] Order type does not have subtotal #2764
[x] Order type does not have total authorized #2763
[ ] Order type does not have total refunded (and neither does the Payment model)
[ ] order lines do not provide a mechanism to ask for a product thumbnail
[ ] payment statuses are not an enum #2744
[x] there are no order event types #2154
[ ] there is a function called Ø that could probably use a better name
[ ] order summary uses fixed-height boxes with all labels grouped in one div and all values in another (this breaks when we need to show additional rows and is terrible for a11y)
[ ] shipping method dialog is named "Edit customer details"
[ ] it's impossible to select anything from the list of shipping methods without typing first
[ ] shipping method selection dialog does not indicate the previous value
[ ] there's no way I found to edit either of the addresses
[ ] order cancellation can be repeated even if order was already cancelled #2808
[ ] there's no UI to select whether stock should be restored on cancellation
[ ] onCustomerEmailClick sounds like it should really just be an <a href="mailto:...">
[ ] fulfillmentCreate(input: { order: ID!, ... }) should be orderCreateFulfillment(order: ID!, input: ...) and should return order #2809
[ ] there's no "success" confirmation of any of the operations, be it a visual cue or an explicit toaster message
[x] orderAddNote should take order: ID! an argument and should return order #2810
[ ] Form > OrderFulfillmentDialog should become OrderFulfillmentDialog > OrderFulfillmentForm > Form
most mutations are not connected:

[x] cancel order
[ ] customer edit - UI is not connected to a callback
[ ] add product - UI is not connected to a callback
change shipping method

[ ] UI is not connected to a callback
[ ] mutation missing #2807

[ ] change shipping address - UI is not connected to a callback
[ ] change billing address - UI is not connected to a callback
[ ] fulfillment - UI is not connected to a callback
[x] capture - UI is not connected to a callback
[x] refund - UI is not connected to a callback
[x] void
[ ] add order note - UI is not connected to a callback

Pull Request Checklist

[ ] Privileged views and APIs are guarded by proper permission checks.
[ ] All visible strings are translated with proper context.
[ ] All data-formatting is locale-aware (dates, numbers, and so on).
[ ] Database queries are optimized and the number of queries is constant.
[ ] The changes are tested.
[ ] The code is documented (docstrings, project documentation).

Very nice work with Ø function! But I feel like it should be introduced in separate PR, since this is project-wide change.

@dominik-zeglen Eventually this function will be used across the whole project, but extracting a separate PR now would only add extra work and dependencies between PRs which slows us down. I think we can as well introduce it incrementally.

Blocked by #2849

Blocked by #2854


Legacy Games library integration issue

Getting this error:
Failed to get Legacy Games games: Value cannot be null. Parameter name: source
Legacy games launcher is installed and I am logged in.
extensions.log
playnite.log

Apparently I forgot to log that exception. My bad. Can you trigger the error again with this version of the plugin and then send me the logs again?
https://github.com/Jeshibu/PlayniteExtensions/releases/download/legacygames0.1.1/LegacyGamesLibrary_0_1_2.pext

extensions.log
playnite.log
Here you go

In your %appdata%\legacy-games-launcher folder there is a file named app-state.json. The plugin reads this for most of its data. In it, there is a user.profile.downloads property, that looks like this (mine is at the end of the file, yours might not be):

Could you share what yours looks like? The error is saying that the downloads property is empty, which is where all your game (bundle) purchases/entitlements are stored.

I checked the file and there was no downloads section, it stopped after username, so I opened the Legacy launcher, downloaded a game and checked the file again and now my file looks just like yours, no more errors in playnite and it imported all my games perfectly.
Seems like you need to download at least 1 game for that section to populate with all your owned games.

Should be fixed now.


Unable to install package on angular 15 (Is there any chance that this package will update?)

You can use this forked version, I guess, https://www.npmjs.com/package/@danielmoncada/angular-datetime-picker. The original was last updated 4 years ago.


TLS flush fix

make sure application receives all available bytes from TLS engine

I'm hoping there's a chance this may fix the windows tests that fail in #94


Fix: Added code for 'Simple Registration Form' (Issue #84)

Related Issue: #84
Previews:

@thepranaygupta @nandini7637 @Abirpal202049 @saikatmu31 Please review this PR.

@nandini7637 Made changes as directed. Please review the changes.


fix: set backup policy template for the comp-definition of apecloud-mysql

fix https://github.com/apecloud/kubeblocks/issues/6992

/cherry-pick release-0.9


Fix COCO configuration

I tried the training and testing on COCO dataset and fixed some minor bugs in the configuration.

Hi @d-li14 I hope you have solved your problem by now, but just for the record you can find other sources of the file as stated in issue #205 and #123

Thanks @ddonatien , but as far as I'm concerned, as stated in #123, just replacing instances_trainval_35k.json with valminusminival2014.json seems not reasonable.

It does not work to me then, the script is asking fot the json file, but it is a zip that cannot be unzipped.

I tried the training on COCO dataset ,can anyone tell me how to modify the coco_labels.txt?

I think  instances_trainval_35k.json is the COCO2017 instances_train2017.json.Beacuse it's have 118287 images.So trainval_35k is just the COCO2017train.

I think :1.trainval35k is COCO 2017 train,because it have 118287 images.
2.So the instances_trainval35k.json is instances_train2017.json

@814247846 Did you get work with that json file?

@814247846 Did you get work with that json file?

yes.But you also have to change:
img = cv2.imread(osp.join(self.root, path))---->img = cv2.imread(path)
I think it's possible that the author made a mistake.

@814247846 all PR are welcomed.

The data sets include 2014 (2014 Train images, 2014 Val images, 2014 Test images), 2015 (2015 Test images) and 2017 (2017 Train images, 2017 Val images, 2017 Test images). Among them, training set and validation set There are tags (2014 Train/Val annotations, 2017 Train/Val annotations), while the test set has no tags.
In the 2014 data set, the training set is 82,783, the validation set is 40504, and the test set is 40775. In addition, the validation set is divided into two parts, miniVal has 5000, and the remaining 35504 images and training set are called Trainval35k (Trainval35k = = train2014 + val2014 - minival2014.). Trainval35k is usually used as the training set in the paper.
There are 81,434 images in the 2015 test set.
In the 2017 data set, the training set is 118287, the verification is 5000, and the test is 40670. Among them, the training set is Trainval35k (82783 + 40504 - 5000 = 118287), and the verification set is miniVal. That is, train2017 == trainval35k == train2014 + val2014 - minival2014 == train2014 + val2014 - val2017.
Original link: https://blog.csdn.net/qq_37643960/article/details/103466904


Issue #609 Create release info file in json format

What this PR does is add a message about available update when the user runs crc start command, and asks if he wants to keep using the current version or stop the current version (to download the newer one).
When a new version is available in mirror.openshift.com, and the user starts crc, he'll get a prompt like following:
❯ crc start -b ~/Downloads/crc_archives/crc_libvirt_4.1.14.crcbundle -p ~/Documents/pull-secret --log-level debug
INFO A new version (1.0.0-beta.6) has been published
Would you like to continue using the current version? [y/N]:

If answered No, it prints the URL to download the latest version of crc. (going forward we'd automatically download the latest version but that is for later.)
❯ crc start -b ~/Downloads/crc_archives/crc_libvirt_4.1.14.crcbundle -p ~/Documents/pull-secret --log-level debug
INFO A new version (1.0.0-beta.6) has been published
Would you like to continue using the current version? [y/N]: n
Please download the new version from https://cloud.redhat.com/openshift/install/crc/installer-provisioned

Note: it'll always interrupt the start once a new version is available
However in https://github.com/code-ready/crc/pull/619#discussion_r328507543 it is mentioned that prompting the user like this, is disruptive, and a WARN msg would be enough.
Fixes #609

Is the goal to have this included in the RC ?

@gbraad Yes, good to have this in, users would be able to know about the next update sooner, before their certificates expire.

Yes, good to have this in,

what was/is not according to schedule, as we have planned/decided to move anything related to update strategy until after the RC/GA

what was/is not according to schedule, as we have planned/decided to move anything related to update strategy until after the RC/GA

I don't remember us actually planning to do this, but since the 4.2 was delayed, we got some more time and its like good to have, not necessary for RC!

good to have

... if strategy and use is clear. since once this is in, we can not just change behaviour.

... if strategy and use is clear. since once this is in, we can not just change behaviour.

strategy wise, these changes are only checking if a new version is released, which we'd always need to know if we want to or need to upgrade. i don't think the actual upgrade strategy is defined by it.

asks if he wants to keep using the current version or stop the current version (to download the newer one).

This means that start will have two different behaverioul paths; no new version and just continues. and one that blocks the user on start. This sounds like a bad feature, as this is not something the user will expect to happen. He might start it and walk away to get coffee, and upon return sees a prompt. A user might do this from a CI script, and the build just hangs awaiting input... or the adapter team can not give input, as they use a non-pty terminal.

Note: it'll always interrupt the start once a new version is available

Please explain once: it will always interrupt the flow when a new version is available? or do we keep state about the version we want to ignore?

When a new version is available in mirror.openshift.com, and the user starts crc, he'll get a prompt like following:

I would say just add warn message with download URI and not stop for any prompt part for crc start we will later introduce crc update command then we can easily change this message by adding crc update at the end. This way it is less destructive for our crc start side also.

This sounds like a bad feature, as this is not something the user will expect to happen.

Right, this will be like a sudden behaviour change, although there's a config option and a flag for crc start to disable it, but if the user doesn't know about it, that doesn't help much. So the other option is to just show a WARN with the update message on start as @cfergeau suggested.

Please explain once: it will always interrupt the flow when a new version is available? or do we keep state about the version we want to ignore?

It'll always interrupt the flow when a new version is available, until it is updated, once it is updated to the latest one, there's no new version so the prompting stops.

Maybe we should only do this once every few days, and stop doing it once we detected a new version is available, but that's additional code that can be done on top of this.

@cfergeau Sure, i can send a follow up patch for these improvements.

how about INFO Version is still current on normal flow, and WARN New version is available when this is the case. This way there is always the same amount of message and the user can see we check for this. Since when there is no connection, I also expect to see a WARN Unable to determine if version is current since the conbnection either failed or the file didn't parse.

@gbraad The messages about Failed checking for new version and Already running the current version is already part of the debug logs as i think these won't be much useful to the user.

retest this please.


Do not lint excluded files

Because our plugin runs on all the files added or modified by the PR, unless a directory is given, we end up ignoring the "excluded" configuration that users might have in the .swiftlint.yml.
This can be problematic in projects that track things like the Carthage/Checkout folder in the repo. Two reasons:

First of all the repos is set to exclude such directories, we should do the same; also
A PR that updates one or more of these might end up with a lot of SwiftLint warnings. This in turn might result in Danger comment making a long comment, exceeding the 65536 characters limit imposed by the GitHub APIs.

I feel this PR should be squashed and merged. I left the full commit history just to show the thought process  ¯_(ツ)_/¯.

I've added you as an owner to the gem, would you like to push a new version? Let me know if you run into issues 👍


Crash compiler  in all 7xx tests during add_hash_columns_mappings_for_sat

line 855, in add_hash_column_mappings_for_sat
f"Parent table '{satellite_parent_table['table_name']}' has no load operation for relation '{relation_name}' requiered for satellite '{table_name}'")

Rework of compiler solves the crash issue


add meta pools votes support to VestedLpMining

Codecov Report

Merging #52 (f7e10ff) into master (a014f31) will increase coverage by 0.10%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #52      +/-   ##
==========================================
+ Coverage   82.89%   82.99%   +0.10%
==========================================
  Files          34       34
  Lines        1964     1976      +12
  Branches      298      299       +1
==========================================
+ Hits         1628     1640      +12
  Misses        336      336

Impacted Files
Coverage Δ

contracts/EthPiptSwap.sol
95.03% <100.00%> (ø)

contracts/balancer-core/BPool.sol
88.43% <100.00%> (ø)

contracts/powerindex-mining/VestedLPMining.sol
89.95% <100.00%> (+0.55%)
:arrow_up:

contracts/powerindex-router/PowerIndexWrapper.sol
75.40% <100.00%> (ø)

contracts/powerindex-router/WrappedPiErc20.sol
93.75% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 984e89c...f7e10ff. Read the comment docs.


SAM-3017 - Changing id to eid in LRS statements, changing to current

site

I can't tell if publishedAssessment.getAssessmentAccessControl().getReleaseTo() would contain the right value all the time or not, so just changed it to be the current site based on the AgentFacade like other code is doing. Also switch id to eid.

LGTM


Page 10053: Service Stats Tax Included Code Issue

On the "Service Stats." page, in the OnAfterGetRecord method, the call to ServAmtsMgt.SumServiceLinesTemp is before the call to SalesTaxCalculate.DistTaxOverServLines. This causes the total that was summed in ServAmtsMgt.SumServiceLinesTemp to be incorrect when sales tax is included in the price. There is code around the SalesTaxCalculate.DistTaxOverServLines code to call ServAmtsMgt.SumServiceLinesTemp after the call to SalesTaxCalculate.DistTaxOverServLines. I believe that the commented out code should be uncommented.

Please report this issue via the standard support channel. This is not related to extensibility.


Update OverlayScrollbars to v2

Currently easy-email uses v1 of OverlayScrollbars. v2 was released recently and has a lot of improvements compared to v2. (Focus on modern browsers, 50% smaller size, treeshaking, deferred initialization and many more).
A full list of features and changes can be found here.

PR is welcome.

Maybe we could let issues like that open an see if we can get contributions


Fixed issue with macOS Catalina and pcre 8.43 enabling pcre-jit

Created issue #6192 to look at this in the future if it is working again.

[approve ci autest]

Potentially a 9.0.x candidate

Cherry-picked to v9.0.x branch.

It looks like this fixed #5243.


Pr/player action packet

Istg I was about the push it myself

I will upstream those changes later manually, thanks for all the help!


Refine the how-to guide so it's cleaner and each new version has a link to the previous version's documentation.

As it is now, the how-to guide is a mess with some stuff that was changed in different versions, so a cleanup would make it (more) readable. Adding a link to the previous version's how-to guide at the bottom with a main page linking to all the versions (like "version 0.1.2 • all versions" or something) would be a good idea so it's easy to keep each version's guide as clean as possible.

This is done. I'll do the "all versions" thing later.


New UI for clear the chrome storage for this extension

It is needed to have a  UI button to clear the whole chrome storage

i think i add a UI like this https://developer.chrome.com/docs/extensions/mv3/user_interface/
i will add one button to clear the storage.


Add feature to provision multiple hosts

I'm using this project frequently, it would be nice if there is field to select how many machines to create.

I added this feature, and I'll will create a PR to add this feature soon :)

Close #2


Update Zend_Version::getLatest()

Zend_Version::getLatest() currently accesses http://framework.zend.com/api/zf-version for the latest ZF1 version.
However, zf1-future has forked the repo and so this is broken.  getLatest() is currently reporting v 1.12.0 while zf1-future is presently at 1.17.0

Thanks, good spot

@joefresco got any ideas for a solution? I had a look at the packagist api and couldn't see anything to solve the issue

A couple possible approaches

Add a text file at the root of the framework (or somewhere else benign) that just contains the version #.  The latestVersion() check could then look up that text file in master on github.
Look up github 'releases' if releases were all named identically and release #s were always the version #.  Docs:  https://developer.github.com/v3/repos/releases/#get-the-latest-release   (name or tag_name would probably be used -- the current name scheme would require splitting the name on the dash and NEVER using the dash as part of the version #.
Example:  https://api.github.com/repos/Shardj/zf1-future/releases/latest
Run a copy of zf1-future/master somewhere that auto-updates with any change to that branch.  I think an 'API' URL could be constructed that would report the version #.

I seem to be getting 403 forbidden response codes back from file_get_contents('https://api.github.com/repos/Shardj/zf1-future/releases/latest') however accessing the url through my browser is no issue at all.

Several options:  https://stackoverflow.com/questions/37141315/file-get-contents-gets-403-from-api-github-com-every-time

Should be working now in release-1.17.1

Thanks!


[WIP] GraphQL query for account transactions

It's a draft of an implementation to initiate discussion, searching for a better way to do it :)

The main concern for me here is that I'm not able to reuse Transaction model because I can't instantiate with a plain json, I need XDR. It seems that our "models" mainly have only the methods to build themselves. So they can be, roughly speaking, replaced with interfaces, and XDR parsing will stand separately. It may facilitate their usage, make them more flexible.
Also is this a good idea to declare interfaces for the data types we store in Dgraph, as I done it with TransactionData?

I rebased this PR on the feature/ingest-script branch to be able to ingest test data locally during development. So this PR should be merged after #64, or we can just discard #64 🙂

So I managed to make account transactions query look a bit better. I created ITransaction interface, which our Transaction model now implements. It allowed me to reuse existing transaction structure in the query without instantiating Transaction model (it was impossible without XDR structures at hand).
Also I got rid of TransactionMemo model in favor of "native" Memo class from stellar-sdk package.
I applied sorting and limiting to this query, however offset parameter in the Dgraph query acts weird. Query looks like this:
{
  txs(func: eq(type, "account")) @filter(eq(id, "GBRP...X2H")) {
    transactions(
      first: 3,
      offset: $offset,
      orderdesc: seq,
      orderdesc: index
    ) {
      id
      seq
      index
    }
  }
}

Here's the result with no offset:
"transactions": [
          {
            "id": "7f947d82de5509f9b2c39fb045815804a2a0f3c400fd1168acc46b72c50ef826",
            "seq": 4,
            "index": 3,
          },
          {
            "id": "fe471dea8d77ae2dfe973829720fe378b3ce53bcef4796f37d43a65487bc0419",
            "seq": 4,
            "index": 2,
          },
          {
            "id": "d079a07a573189a17ad42ddcff9ee609efbb323fc7ee4fabb2a97d4cde7994a4",
            "seq": 4,
            "index": 1,
          }
]

And here with offset = 1:
"transactions": [
          {
            "id": "7f947d82de5509f9b2c39fb045815804a2a0f3c400fd1168acc46b72c50ef826",
            "seq": 4,
            "index": 3
          },
          {
            "id": "fe471dea8d77ae2dfe973829720fe378b3ce53bcef4796f37d43a65487bc0419",
            "seq": 4,
            "index": 2
          },
          {
            "id": "2541db914d8c49e8f10e84f3f2f13a9ba63bb112f602a6411044b7fc37d96070",
            "seq": 3,
            "index": 3
          }
        ]

For some reason the first two transactions stay at their places, and only third is moved. It's not, what I expect from offset 🙂 I think, it's somehow related to sorting. I will investigate it further. Or maybe @gzigzigzeo you have any ideas?

The issue with offset, described above, is resolved by using order field for sorting. Thanks, @gzigzigzeo for pointing it out! 👍

Since I messed up merging chore/dgraph branch into master, I have to close this PR and put these changes on master manually 😞


Reformat the README

We don't need all the descriptions for each of the dependencies. There are also some spelling errors through out the document that need to be fixed.

Fixed


FARM-Beta validator

Pull Request Description
What issue does this change request address?
FARM based validator is implemented.

FARM-Beta is the new version ready for production phase; Also it will be the ground brick for FARM-Gamma that will be developed by April 2022.
FARM-Alpha is the prototype where everything is hard-coded. FARM-Alpha can be deleted to save reviewer's time.

What are the significant changes in functionality due to this change request?
The FARM-Beta validator is implemented in this pull request. FARM-Beta recognizes the user-provided IES list of components and system configurations from the HERON input file, and perform the setpoint validation during the power dispatch optimization so that the system implicit operational constraints are not violated during transients and steady-state operations.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[ ] 1. Review all computer code.
[ ] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[ ] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass.
[ ] 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large tes.
[ ] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[ ] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[ ] 8. If an analytic test is changed/added, the the analytic documentation must be updated/added.
[ ] 9. If any test used as a basis for documentation examples have been changed, the associated documentation must be reviewed and assured the text matches the example.

@dylanjm @PaulTalbot-INL Hey Dylan and Paul, would you place the review work on hold for one day or two? I will let you know when I addressed your comments in one day or two. We are overwhelmed by the January Milestone this week. Thanks, Haoyu

@PaulTalbot-INL @dylanjm Hey Dylan and Paul, I have deleted the generated files and basically all the "print" commands. Also, the "_utils.py" import has been fixed.
However I saw both the Precheck and CentOS 8 check has failed. Could you share with me the error messages? When I click the "Details" link it says "You are not allowed to see results. Please sign in to an appropriate user."
Thanks! Haoyu

@PaulTalbot-INL @dylanjm Hey Dylan and Paul, I have deleted the generated files and basically all the "print" commands. Also, the "_utils.py" import has been fixed.
However I saw both the Precheck and CentOS 8 check has failed. Could you share with me the error messages? When I click the "Details" link it says "You are not allowed to see results. Please sign in to an appropriate user."
Thanks! Haoyu

I assume you tried signing in using GitHub? Does it still not show you results?
CentOS 8 failed because precheck failed, and precheck failed with:
##########################################################################
ERROR: The following files contain trailing whitespace after applying your patch:
	src/validators/FARMValidators.py
	tests/integration_tests/mechanics/validator_farm_beta/transfers.py

Run the "delete_trailing_whitespace.sh" script in your $MOOSE_DIR/scripts directory.
##########################################################################

ERROR: exiting with code 1

//: Singularity execution completed with return code 1

@PaulTalbot-INL Thank you Paul for the information. You are correct, I have signed in on GitHub, but it doesn't show me the results.
I ran the "delete_trailing_whitespace.sh" on these two files and now the precheck is passed.
Thanks,
Haoyu

Superseded by #150.


[Docker Hub] Make it possible to link directly to subsections in image description docs

Tell us about your request
I'd like to be able to link to paragraphs in image docs, just like I can in a Markdown file in GitHub.
To point someone directly at the section of the documentation, in GitHub I'd browse to the appropriate spot, hit the 🔗(link) icon, and send them the link. For example: https://github.com/docker-library/docs/tree/master/caddy#adding-custom-caddy-modules
In DockerHub, I'd like to be able to do the same, but currently all I can do is send them https://hub.docker.com/_/caddy?tab=description and ask them to ⌃F and search the page for Adding custom Caddy modules.
Which service(s) is this request for?
DockerHub
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
It's frustrating to send someone the link to a DockerHub page to point them at a piece of documentation, and not be able to send the exact link they need.
Are you currently working around the issue?
Not in any satisfactory way!
Additional context
n/a

For official images at least, a workaround is to link to the GitHub copy of the rendered description instead: https://github.com/docker-library/docs/tree/master/caddy#adding-custom-caddy-modules
(Although I fully agree this isn't a great workaround and should be possible on Hub instead :innocent: :heart: :+1:)

The URL should also always default to description instead of ?tab=description. Ideally the query string should be replaced with actual URLs. So
https://hub.docker.com/_/caddy  -> shows description
https://hub.docker.com/_/caddy/reviews -> shows reviews
https://hub.docker.com/_/caddy/tags -> shows tags

The URL should also always default to description instead of ?tab=description

True - sorry if my description was misleading, I wanted to clarify that I'd be sending them to the description tab specifically.
Clicking on the description tab takes you to ?tab=description - the query string seems to not be replaced for any of the other URLs either. Though that's somewhat orthogonal to this issue 😉.

@pkennedyr any updates on this?

cc @nebuk89


How to test TOAST application on emulator

Hello.
Please help!
Note: Developed on Mac OSX High Sierra
I just finish to develop a TOAST application which is a News channel for the major News channel in Mexico. (TELEVISA)  I've been testing it on browser by following the docs in here: https://github.com/Samsung/cordova-plugin-toast/wiki/Create-Project
https://github.com/Samsung/cordova-plugin-toast.
I've been able to run my application successfully with no problems at all when it comes to browser. But this application is intended to work on real devices from Orsay to Tizen platforms. So, I've followed the instructions for prepare and build for those platforms but my problem comes when I try to run my TOAST app on an emulator, say Orsay or TIZEN.
When I try to prepare my application for either Orsay or Tizen with the command "$ grunt sectv-prepare:sectv-" I always get this error:
"UnhandledPromiseRejectionWarning: TypeError: js2xmlparser is not a function".
I've tried to upgrade js2xmlparser to the latest version but still get that error.
Anyway, just for testing purposes, I've continued with the prepare and build process, and for Orsay platform, I've been able to compile the app in a .zip after ran the command "$ grunt sectv-build:sectv-orsay"
And then I've tried to run that .zip file on eclipse IDE for MAC OSX but there is no way to test it on a samsung Tv emulator. I've downloaded Virtualbox in order to run the emulator, but I got no luck.
Pease help me understand how is the process to emulate a TOAST application from the SDK as mentioned in the docs: # Run the application on target by using SDK.
Thanks in advance
NOTE: I've downloaded and intalled both SDKs: Tizen Studio and Eclipse IDE

Hi there, please refer similar issue on grunt-cordova-sectv first.
If you can't resolve that problem still doing like that, Please let me know again.
https://github.com/Samsung/grunt-cordova-sectv/issues/14
Thanks.

Thanks for your quick response. I've solved the issue with js2xmlparser but I'm still stuck. Right now I can prepare and build the project for Orsay. The Grunt task generate a .zip file under /myapp/platforms/sectv-orsay/build/.
I've imported my root project folder into Eclipse IDE and everything seems ok. I have the project tree in Eclipse IDE, but I can't figure out how to test it on an emulator for Samsung Tv.
TIZEN
For Tizen Platform I get this error:
Fatal error: The command "tizen" failed. Make sure you have the latest Tizen SDK installed, and the "tizen" command (inside the tools/ide/bin folder) is added to your path.
I've downloaded Tizen SDK from here: https://developer.samsung.com/tv/develop/tools/
and followed the installation instructions. And I've added the tizen command with this: export "PATH=$PATH:$<TIZEN_STUDIO>/tools/ide/bin/" but it seems like tizen cli is not installed at all. When I try $ tizen command o get command not found.
Please help me to figure out how to test my application on emulators as mentioned on the docs:
https://github.com/Samsung/cordova-plugin-toast
Thanks in advance

Hi. Thanks for your quick response.
Yesterday I'found the solution for the js2xmlparse error. After fix it I was able to prepare the application for both Orsay and Tizen platforms, but still have no clue how to test it on the emulator for neither Orsay nor Tizen.
ORSAY
After build the application with grunt sectv-build:sectv-orsay I get the .zip file in /myapp/platforms/sectv-orsay/build/. I think that is ok. Now. In Eclipse IDE I was able to import the project. But still I have no idea how to run the project on an emulator.
I've downloaded the Samsung TV SDK for Legacy Platform (Emulator Image for Virtual Box) which is a .ova file. I've imported it into Virtualbox and now I have the emulator running from Virtualbox but now, I think my problem is how to launch the emulator from Eclipse IDE with my application.
Please help me how to launch application from Eclipse IDE in a Samsung TV emulator.
TIZEN
After run the command for build the application for Tizen platform grunt sectv-build:sectv-tizen
I get the following message from terminal:
Running "sectv-build:sectv-tizen" (sectv-build) task
Start packaging Samsung Tizen TV Platform......
Tizen CLI 2.4.50
Setting configuration is succeeded....
Build Web Application:   80% ( 8/10)There is no /holkanluna/tizen-workspace/.metadata/.plugins/org.tizen.common.sign/profiles.xml.
There is no hkDevProfile.
Warning: Not found tizen signature file
Warning: Not found tizen author signature file
Excludes File Pattern: {.manifest.tmp, .delta.lst}
Fatal error: Cannot read property 'match' of undefined
I've created the hkDevProfile yesterday through TizenStudio > certificate manager.
What I'm doing wrong?
Thanks in advance!

Hi,
You mean you want to launch your Toast application for each emulators.
Please below guides for launching application on Emulator.
Tizen (14'~)
https://developer.samsung.com/tv/develop/getting-started/using-sdk/tv-emulator/
https://developer.samsung.com/tv/develop/getting-started/setting-up-sdk/creating-certificates
https://developer.samsung.com/tv/develop/getting-started/creating-tv-applications
Before Tizen(~13')
https://developer.samsung.com/tv/develop/legacy-platform-library/d04/index#Run-and-Debug-App-(SDK-5.0)
Even though you can't use "build" command because of any errors,
If you succeed to "prepare" command, you can just import your application on your each SDK such as the other common TV web application.
In case of your issue on Tizen, It might be a creating certification was failed.
First, please check to be able to launch an new empty project, not Toast.
If you have still in trouble with importing your empty application on SDK, It might not be an issue about Toast itself.
I recommend to you to contact samsung developer site for asking that.
https://developer.samsung.com/community
Thanks!

Hi. Well, I've read and followed all those links you gave me.
I've imported the TOAST project into TizenStudio successfully after grunt sectv-prepare:sectv-tizen and I've followed the instructions for run/debug the project as > Tizen Web Application in the emulator, but I get the following errors on console about missing/Not found files (cordova_plugins.js as well as the "plugins" folder).

But here https://github.com/Samsung/cordova-plugin-toast/issues/17/#issuecomment-224260470, Sue comments that those files and folders are not required on a Tizen/Orsay platforms.
Then, why the console complains of this?
Also, the console complains about toast not defined.

Let me know if I can send you my TOAST project for you to check if there are any errors or something that I've missed. I've tested this TOAST application on browser an it works as expected.
Please help us here. We are kind of stuck.
Thanks in advance.

Ok, before you sending me an app, Please check this first.
Did you add a code for adding toast.js in index.html?
Please refer below guide.
Even though you check it all, you've got same error
Please send me an application. Then I will check it.
https://github.com/Samsung/cordova-plugin-toast/wiki/Create-Project#wwwindexhtml
Thanks.


TX made from Samurai Paynym to Sparrow Paynym not received

Made a TX from SamuraiWallet to SparrowWallet Paynym Contact - tx has 12 confirmations but Sparrow does not show any income :-(

Is this the same PayNym from #474?

Also, what method of connection are you using with Sparrow - Bitcoin Core or Electrum server?

Also, what method of connection are you using with Sparrow - Bitcoin Core or Electrum server?

Electrum Server and yes it is the same paynym like 474.

My log is full of these ERRORs... :-(
022-03-20 21:21:53,490 ERROR [JavaFX Application Thread] c.s.s.w.WalletForm [null:-1] Could not determine payment codes for wallet SparrowWallet
java.lang.IllegalStateException: Cannot add payment code wallet to P2TR wallet
at com.sparrowwallet.drongo/com.sparrowwallet.drongo.wallet.Wallet.addChildWallet(Unknown Source)
at com.sparrowwallet.drongo/com.sparrowwallet.drongo.wallet.Wallet.addChildWallet(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$PaymentCodesService$1.call(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$PaymentCodesService$1.call(Unknown Source)
at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$6(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Unknown Source)
at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$7(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
2022-03-20 21:21:53,949 ERROR [JavaFX Application Thread] c.s.s.w.WalletForm [null:-1] Could not determine payment codes for wallet SparrowWallet
java.lang.IllegalStateException: Cannot add payment code wallet to P2TR wallet
at com.sparrowwallet.drongo/com.sparrowwallet.drongo.wallet.Wallet.addChildWallet(Unknown Source)
at com.sparrowwallet.drongo/com.sparrowwallet.drongo.wallet.Wallet.addChildWallet(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$PaymentCodesService$1.call(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$PaymentCodesService$1.call(Unknown Source)
at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$6(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Unknown Source)
at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$7(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
2022-03-20 21:21:54,621 ERROR [JavaFX Application Thread] c.s.s.w.WalletForm [null:-1] Could not determine payment codes for wallet SparrowWallet
com.sparrowwallet.sparrow.net.ServerException: Failed to retrieve transaction history for paths: [m/0]
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer.getReferences(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer.getHistoryToGapLimit(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer.getHistory(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$PaymentCodesService$1.call(Unknown Source)
at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$PaymentCodesService$1.call(Unknown Source)
at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$6(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Unknown Source)
at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$7(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.sparrowwallet.sparrow.net.ServerException: Retries exhausted
at<EMAIL_ADDRESS>

Thanks for the logs.
It appears you are using a Taproot (P2TR) wallet. PayNyms are not currently supported in P2TR wallets - Sparrow should have done a better job to disable BIP47 functionality, and this has been fixed in https://github.com/sparrowwallet/drongo/commit/88bd6ada98d98451c580ee4209ec86ec95fa2c94. I've also updated the documentation for this. Taproot wallets will probably be supported in future, but the spec will need to be extended for this.
The good news is that the funds are not lost. You can create a Native Segwit wallet in Sparrow with the same seed words to recover them. You may need to send another notification transaction (it depends if one was already sent from the Samourai +fancyunion929 wallet) but once the link is established Sparrow should find the previously sent transaction.

Well, after recovering the wallet I get a new/different paynym ID of course - it does not find the previous transactions.

You should not get a new PayNym, since the payment code for the wallet is constructed independently of the script type.
Check that the master fingerprint of the Native Segwit wallet matches that of the Taproot wallet - if they differ, the seed is different.

Negative. I still have access to the Taproot Wallet and can display my BIP-39 seed. Using exactly this seed for creating a new keystore and importing it into a new wallet I get a different fingerprint (!).

Are you using a passphrase in either wallet?

Yes, I do.

Different passphrases are the only explanation here - make sure the passphrase is the same in the Taproot and Native Segwit wallets. If the passphrase and the seed words are identical, the master fingerprint and PayNym will be too.

User to dumb error - mixed up password and passphrase ... thanks for your help!


Seq2Seq CodeBERT - ignore index of CE loss = -1

Dear CodeXGLUE team,
Thanks for providing such an amazing benchmark!
Given the pad token id of CodeBERT tokenizer is "1"
I'm wondering why the ignore_index param in the CrossEntropyLoss is set to "-1" in the following code
https://github.com/microsoft/CodeXGLUE/blob/d1e2f6ce4ea7d7280a0d21178f36cde5ee830929/Code-Code/code-to-code-trans/code/model.py#L68
Kind regards
Michael

This ignore_index=-1 has no effect, because we have use active_loss to mask the padding token in loss calculation here https://github.com/microsoft/CodeXGLUE/blob/d1e2f6ce4ea7d7280a0d21178f36cde5ee830929/Code-Code/code-to-code-trans/code/model.py#L64

Thanks for your swift response! @celbree
Just to confirm that
active_loss = target_mask[..., 1:].ne(0).view(-1) == 1
the "0" here in the .ne() is the pad token id, am I right?

Actually the 0 in target_mask is not pad token id, but indicts pad tokens in the same positions in target_ids. Please refer to these lines.
https://github.com/microsoft/CodeXGLUE/blob/d1e2f6ce4ea7d7280a0d21178f36cde5ee830929/Code-Code/code-to-code-trans/code/run.py#L133
https://github.com/microsoft/CodeXGLUE/blob/d1e2f6ce4ea7d7280a0d21178f36cde5ee830929/Code-Code/code-to-code-trans/code/run.py#L134

I see
so anyway the result of active_loss should be something like [True, True, True, False, False] if I have target like [token, token, token, , ], is this correct?

That's true!

Thank you very much :) !


MINOR: clarify partition option behavior for console consumer

Console Consumer help doesn't say that --partition option needs --offset otherwise will consume from the end of the partition. This minor fix makes that happen.

Can you change the target branch to trunk please?

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5383/
Test PASSed (JDK 8 and Scala 2.12).

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5398/
Test PASSed (JDK 7 and Scala 2.11).

@ijuma retargeted to trunk and flipped the wording based on your suggestion with one minor addition. Thanks!

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.11/5403/
Test PASSed (JDK 7 and Scala 2.11).

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/5388/
Test PASSed (JDK 8 and Scala 2.12).


Development

Bug fix: classification state and floor map large file upload
Bug fix: Only show readers for the same facility and floor as the project
Added ability to detect and display reader engagement:

which readers are used in the running job?
which readers are connected to other sources? (e.g. octane or other itemsense instances)

Enhanced Zone Editing

Direct editing of X and Y values of points in the UI
Changing the current Zone Map via menu
CRUD on ZoneMaps and Projects

Added Babel to gulpfile so development can continue using ECMA6 syntax
Added Export menu to dump running job or instance configuration to a json file
Added support for getting Item location information from node-red flows
Removed post processing from presentation mode. post processing should now be done in node-red flows.

Looks good!
One question I have is are you planning on migrating everything to es6? Seems odd having some files with half-done with es6 syntax.

It is a big code base. we will add any new code as ES6, and gradually
refactor the technical debt as time permits.
sounds good?
On Tue, Apr 19, 2016 at 9:05 AM, Joe Combopiano<EMAIL_ADDRESS>wrote:

Looks good!
One question I have is are you planning on migrating everything to es6?
Seems odd having some files with half-done with es6 syntax.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/impinj/itemsense-viztool-js/pull/9#issuecomment-211996675


Null able logger in SpassReprintModule

i think logger in SpassReprintModule should be null able.

Reprint will never pass a null instance of a logger into a module. I designed it that way specifically so that modules don't have to do a null check every time they want to log something. Why do you think that should change?

https://github.com/ajalt/reprint/blob/master/reprint_spass/src/main/java/com/github/ajalt/reprint/module/spass/SpassReprintModule.java#L186
so this will not cause problem is logger is null?

sample code.
Reprint.initialize(this);
Reprint.registerModule(new SpassReprintModule(this,null));

Yes, if you manually create an instance of the module and pass in null to a non-nullable parameter, it will cause an NPE. But Reprint.initialize will create and register the SpassReprintModule for you. There's no need to register in manually.

thanks for help :)


fix typo and current path logic

Fix typo
Changed the logic to get model current path

Thanks for your PR🦉


[GPU/OpenCL] Can use at most 8.3GB GPU memory

Hi, thanks for the package! I noticed a similar issue as #3899. I am using LightGBM with a version of 3.2.1, and an NVIDIA Tesla V100-SXM2-16GB (16GB memory). By running the following code, we can see the first run only takes 8299MB GPU memory, which means the second one should fit, since the number of data points only slightly increases. However, it turns out we get the following error message. Could someone let me know if there is an internal memory limit in the LightGBM library? Thanks very much for the help!
Code:
import lightgbm as lgb
import numpy as np

if __name__ == '__main__':
    for n_data in [1040000, 1100000]:
        n_feature = 8000
        print(n_data, n_feature)
        x = np.random.rand(n_data, n_feature)
        y = np.random.rand(n_data)
        model = lgb.LGBMRegressor(device_type='gpu')
        model.fit(X=x, y=y, eval_set=[(x, y)], verbose=1)

Error message:
terminate called after throwing an instance of 'boost::wrapexcept<boost::compute::opencl_error>'
  what():  Memory Object Allocation Failure

Got the same issue.
I'm using C API. When the training dataset is too large, the program would just crash when I call 'LGBM_BoosterCreate()'
With 7.5GB data, it works fine:
[LightGBM] [Info] This is the GPU trainer!!
[LightGBM] [Info] Total Bins 830854
[LightGBM] [Info] Number of data points in the train set: 2413300, number of used features: 3273
[LightGBM] [Info] Using requested OpenCL platform 0 device 0
[LightGBM] [Info] Using GPU Device: NVIDIA GeForce RTX 3090, Vendor: NVIDIA Corporation
[LightGBM] [Info] Compiling OpenCL Kernel with 256 bins...
[LightGBM] [Info] GPU programs have been built
[LightGBM] [Info] Size of histogram bin entry: 8
[LightGBM] [Info] 3273 dense feature groups (7539.72 MB) transferred to GPU in 2.319649 secs. 0 sparse feature groups
If I add more data (estimate ~8.9GB), it would crash before the last line:
[LightGBM] [Info] This is the GPU trainer!!
[LightGBM] [Info] Total Bins 939820
[LightGBM] [Info] Number of data points in the train set: 2413300, number of used features: 3702
[LightGBM] [Info] Using requested OpenCL platform 0 device 0
[LightGBM] [Info] Using GPU Device: NVIDIA GeForce RTX 3090, Vendor: NVIDIA Corporation
[LightGBM] [Info] Compiling OpenCL Kernel with 256 bins...
[LightGBM] [Info] GPU programs have been built
[LightGBM] [Info] Size of histogram bin entry: 8

Crash here. No log any more

And it happens either when I increase number of features or number of data points, as long as the total size is more than ~8GB.
I'm using
LightGbm v3.3.1 C API
CUDA 11.5.56
RTX 3090 with 24GB memory

I have this same problem!!
My company has 2 types GPU: V100-32G and A100-40G
When the X_train is more than shape=(850w, 1000), LGB-GPU will have the same problem. It seems like OOM but I'm not sure. The GPU memory usage about 8.3G same as you.

@Anstinus

I verified it on my machine with more than 9GB GPU memory and it worked smoothly.

Thank you so much for getting back and sharing this observation!

May I ask which release has this fix? I use v3.3.5 version, the problem can still be reproduced

It will be in release v4.0.0. You can follow #5153 to be notified when that release is published.


Additional disk storage not an option in terraform

What happened?
Found the upgrade additional disk storage in the Aiven UI however I do not see that as an option in the terraform docs.
What did you expect to happen?
I would expect to see an additional_disk_storage option in terraform. In particular the pg resource https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/pg

Thank you very much @Serpentiel I really appreciate the quick response on this!

no problem, you're welcome!


[DNM] Add counter example for Nexys Video

This PR adds counter example for Nexys Video board.
Requested in #63

Writing Implementation FASM: top.fasm
/home/runner/opt/symbiflow/xc7/install/bin/vpr_common: line 116:  3703 Killed                  genfasm ${ARCH_DEF} ${EBLIF} --device ${DEVICE_NAME} ${VPR_OPTIONS} --read_rr_graph ${RR_GRAPH} $@
make: *** [Makefile:44: build/top.fasm] Error 137
Error: Process completed with exit code 2.

The process was killed due to high RAM memory consumption. GitHub Actions provide 7GB of RAM on the runners:
https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources
The current example consumes about ~8GB on my local machine. It seems that either some optimization is needed or we need to provide a more advanced CI setup to be able to add Nexys Video examples.
FYI @mithro

We can use Kokoro for these if needed...

added with #118

added with #118


[DataFrame] Parallel Load into dataframe

Is your feature request related to a problem or challenge?
When loading data into a DataFusion via SessionContext::read_parquet, DataFrame , only a single core is used even when there are many cores available.
This leads to slower performance, as reported by @mispp on https://github.com/apache/arrow-datafusion/discussions/6908
Reproducer
Create data using
cd datafusion/benchmarks
./bench.sh data tpch10

Then lad the
use std::{io::Error, time::Instant};
use datafusion::prelude::*;
use chrono;

const FILENAME: &str = "/Users/alamb/Software/arrow-datafusion/benchmarks/data/tpch_sf10/lineitem/part-0.parquet";

#[tokio::main]
async fn main() -> Result<(), Error> {
    env_logger::init();
    {
        let _ = _datafusion().await;
    }

    Ok(())
}

pub async fn _datafusion() {
    let _ctx = SessionContext::new();

    let _read_options = ParquetReadOptions { file_extension: ".parquet", table_partition_cols: vec!(), parquet_pruning: None, skip_metadata: None };
    let _df = _ctx.read_parquet(FILENAME, _read_options).await.unwrap();

    let start = Instant::now();
    println!("datafusion start -> {:?}", chrono::offset::Local::now());

    let _cached = _df.cache().await;
    let elapsed = Instant::now() - start;
    println!("datafusion end -> {:?} {elapsed:?}", chrono::offset::Local::now());
}

Cargo.toml

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[package]
name = "perf_test"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
env_logger = "0.10.0"

parquet = "40.0.0"
serde = "1.0.163"
serde_json = "1.0.96"
datafusion = "27.0.0"
tokio = "1.0"
chrono = "0.4.26"

Describe the solution you'd like
I would like datafusion to read the parquet file in parallel, using target_partitions config parameter
https://docs.rs/datafusion/latest/datafusion/config/struct.ExecutionOptions.html#structfield.target_partitions
Describe alternatives you've considered
No response
Additional context
No response

@alamb Hello, I'm new to DataFusion, but can I give this issue a try?

Thank you @gobraves  -- that would be great. Once you have looked around let me know if you have any questions
Basically I would suggest first verifying that running the equivalent SQL with datafusion-cli
create table t;
INSERT INTO t from SELECT * from `data.parquet`

is properly parallelized
Then look at the plan that comes out
INSERT INTO t from SELECT * from `data.parquet`

And try to update DataFrame::cache() to use the same pla

hi @alamb, I apologize for the delayed response. Based on your tips, I executed the following commands in the CLI and also ran the code you provided to reproduce the issue. I noticed that executing the commands in the CLI was almost 8 times faster than running the code mentioned above, which is consistent with my CPU core count.
Here are the commands I executed in the CLI:
create external table test stored as parquet location 'part-0.parquet';
create table t as select * from test;
explain create table t as select * from test;

In the logical_plan of the explain output, I observed CreateMemoryTable and TableScan. Consequently, I reviewed the code for CreateMemoryTable in the datafusion-cli and the .cache()  function, hoping to identify the differences. I noticed that the target_partitions are indeed passed in both cases, but I'm unsure why they are not utilized in .cache(). However, from the commit mentioned in issue #6984 , it seems that the problem is resolved by using repartitioning. Therefore, it appears that the difference lies in one implementation using Partitioning, while the other does not. However, when browsing through the code myself, I couldn't find any relevant settings. If this is the case, could you please provide some hints as to which part of the code this operation occurs?
I have one more question: Do we need to create a new DmlStatement to address this issue?

Perhaps this could be done by creating a LogicalPlan::DmlStatement for write and then letting the existing insert machinery work rather than doing a custom "collect".

I'm not entirely clear about this statement, and I believe it might be because I haven't fully grasped the problem described above.

@gobraves Thank you for trying! I also took a look at this issue (and find it pretty difficult to solve 😨 ), hope the following info might be helpful:
Here is an overview of parallel parquet scan
let _df = _ctx.read_parquet(FILENAME, _read_options).await.unwrap();

let _cached = _df.cache().await;

After _ctx.read_parquet(...,  a LogicalPlan with TableScan is created and stored inside the dataframe.
Inside _df.cache(), the LogicalPlan will first be converted into a physical plan with ParquetExec node, and then the physical optimizer will try to modify the ParquetExec node's file_groups to make it parallel.
My reproducer:
DataFusion CLI v28.0.0
❯ create external table test stored as parquet location '/Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet';
0 rows in set. Query took 0.064 seconds.
❯ create table t as select * from test;
0 rows in set. Query took 16.364 seconds.
❯ create table t as (select * from test where l_orderkey > 0);
0 rows in set. Query took 3.646 seconds.

❯ explain select * from test;
+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | TableScan: test projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment]                                                                                |
| physical_plan | ParquetExec: file_groups={1 group: [[Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet]]}, projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment] |
|               |                                                                                |
+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.009 seconds.
❯ explain select * from test where l_orderkey > 0;
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | Filter: test.l_orderkey > Int64(0)                                                                                |
|               |   TableScan: test projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment], partial_filters=[test.l_orderkey > Int64(0)]                                                                                |
| physical_plan | CoalesceBatchesExec: target_batch_size=8192                                                                                |
|               |   FilterExec: l_orderkey@0 > 0                                                                                |
|               |     ParquetExec: file_groups={12 groups: [[Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet:0..13271296], [Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet:13271296..26542592], [Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet:26542592..39813888], [Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet:39813888..53085184], [Users/yongting/Desktop/code/my_datafusion/arrow-datafusion/benchmarks/data/tpch_sf1/lineitem/part-0.parquet:53085184..66356480], ...]}, projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment], predicate=l_orderkey@0 > 0, pruning_predicate=l_orderkey_max@0 > 0 |
|               |                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

The 2nd one is parallelized, explain verbose select ... can be used to see the specific physical optimizer rule to repartition the ParquetExec
https://github.com/apache/arrow-datafusion/blob/a9561a0f06c25f370dc39df08d057db85c4e0c7a/datafusion/core/src/physical_optimizer/repartition.rs#L166
I think there might be some bug inside this function, if parquet_exec.get_repartitioned() inside it gets called, then the ParquetExec should be properly parallelized
This reproducer should have the same root cause as the original one, for the original reproducer, adding a filter to _df can also get it parallelized:
    let _df = _ctx
        .read_parquet(FILENAME, _read_options)
        .await
        .unwrap()
        .filter(col("l_orderkey").gt(lit(0)))
        .unwrap();
// Then can be parallelized

I made a POC on #6984 which demonstrates the issue is indeed using more cores to do the write. However, the implementation of doing repartitioning is probably not right -- I think the better approach would be to set the target partitions when writing into memory table

This POC and adding predicate seem both suppress the physical optimizer bug in repartition rule by adding another execution node on top of ParquetExec node 🤔

@2010YOUY01 Thank you!
Based on your findings, I also retested the datafusion-cli and arrow-datafusion crate after updating them to version 28.0.0. Here are the results:

DataFusion CLI v28.0.0
❯ create external table test stored as parquet location 'part-0.parquet';
0 rows in set. Query took 0.023 seconds.
❯   create table t as select * from test;
0 rows in set. Query took 14.621 seconds.

DataFusion CLI v28.0.0
❯ create external table test stored as parquet location 'part-0.parquet';
0 rows in set. Query took 0.014 seconds.
❯  create table t as (select * from test where l_linenumber > 0);
0 rows in set. Query took 4.280 seconds.

use chrono;
use datafusion::common::DataFusionError;
use datafusion::prelude::*;
use object_store::local::LocalFileSystem;
use std::{sync::Arc, time::Instant};
use url::Url;

const FILENAME: &str =
    "/home/neo/project_learning/arrow-datafusion/benchmarks/data/tpch_sf10/lineitem/part-0.parquet";

#[tokio::main]
async fn main() -> Result<(), DataFusionError> {
    let _ctx = SessionContext::new();
    let config = _ctx.copied_config();
    for item in config.options().entries().iter() {
        let key = &item.key;
        let value = &item.value;
        println!("{key} {value:?}")
    }
    let local = Arc::new(LocalFileSystem::new());
    let local_url = Url::parse("file://local").unwrap();
    _ctx.runtime_env().register_object_store(&local_url, local);

    let _read_options = ParquetReadOptions {
        file_extension: ".parquet",
        table_partition_cols: vec![],
        parquet_pruning: None,
        skip_metadata: None,
    };
    let _df = _ctx
        .read_parquet(FILENAME, _read_options)
        .await
        .unwrap();

    let start = Instant::now();
    let _cached = _df.cache().await;
    let elapsed = Instant::now() - start;
    println!(
        "datafusion end -> {:?} {elapsed:?}",
        chrono::offset::Local::now()
    );
    Ok(())
}

without filter:<PHONE_NUMBER>35s
If the code is modified with filter
 let _df = _ctx
        .read_parquet(FILENAME, _read_options)
        .await
        .unwrap()
        .filter(col("l_linenumber").gt(lit(0)))
        .unwrap();

with filter: 15.583268924s

datafuison-cli
script

without filter
14.621s
114.913562535s

with filter
4.280s
15.583268924s

I will need to continue examining the code to understand the specific reason behind this performance difference.

I think there is something in the physical planning that assumes the result of the final plan should be in a single partition (or at least it won't expand it when adding additional partioning). because when connecting to a client this is what makes the most sense
I believe this is controlled by ExecutionPlan::benefits_from_input_partitioning: https://github.com/apache/arrow-datafusion/blob/6a2d4a3a254c0495a398608d178496a191450750/datafusion/core/src/physical_plan/mod.rs#L169-L183
ProjectionExec returns false if it is only columns (which is what these queries are doing)
https://github.com/apache/arrow-datafusion/blob/6a2d4a3a254c0495a398608d178496a191450750/datafusion/core/src/physical_plan/projection.rs#L285
So the reason the filter case goes faster is that the filter is that the filter will return true for benefits from repartitioning but the Partition won't.
I wonder if we could somehow add a flag to LogicalProjection / ProjectionExec that says "always benefits from repartitioning somehow and set that flag for the writes 🤔
Alternately, I was thinking the `ExecutionPlan that does the writing could say "I want the input partitioned" and the optimizer would do the right thing.  But given the DataFrame API doesn't use an ExecutionPlan for writing it might not work.
Thank you both for pushing on this -- it is going to be awesome to get this working correctly

@alamb
I ran the same reproducer as stated here: https://github.com/apache/arrow-datafusion/issues/6983#issuecomment-1662556865 and I can report the same results as before (CLI & Dataframe). The issue seems not to be resolved, unfortunately.

Thank you for the follow up @marvinlanhenke

Possible Solution:
Simply remove the implementation on OutputRequirementsExec and also rely on the default impl from the trait?

I think this is likely a great thing to try. @devinjdangelo perhaps you have some more input or ideas to try

@ alamb
I did remove fn benefits_from_input_partitioning and it works just fine.
Now, I get file_groups={12 groups:...} for  OutputRequirementsExec without having a Filter-, or AggregateExec on top.
...I'll push that minor change for now; if there is not another solution that is more suitable.

I ran the reproducer https://github.com/apache/arrow-datafusion/issues/6983#issuecomment-1662556865 and didn't see this issue.

generate benchmark data

cd benchmarks
./bench.sh data tpch10

run CLI with query (3.2 seconds) and without query (3.5 seconds)

DataFusion CLI v36.0.0
❯ create external table test stored as parquet location '/Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet';
0 rows in set. Query took 0.115 seconds.

❯ create table t as select * from test;
0 rows in set. Query took 3.527 seconds.

DataFusion CLI v36.0.0
❯ create external table test stored as parquet location '/Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet';
0 rows in set. Query took 0.006 seconds.

❯ create table t as (select * from test where l_linenumber > 0);
0 rows in set. Query took 3.216 seconds.

ran the rust program with query (3.1 seconds) and without query (3 seconds)

    let _df = _ctx
        .read_parquet(FILENAME, _read_options)
        .await
        .unwrap();
        // .filter(col("l_orderkey").gt(lit(0)))
        // .unwrap();

checked the plan output for the presence of file_groups in the physical plan to make it parallel.

❯ explain select * from test;
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | TableScan: test projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment]                                                                                |
| physical_plan | ParquetExec: file_groups={4 groups: [[Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:0..10165445], [Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:10165445..20330890], [Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:20330890..30496335], [Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:30496335..40661778]]}, projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment] |
|               |                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.010 seconds.

❯ explain select * from test where l_orderkey > 0;
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | Filter: test.l_orderkey > Int64(0)                                                                                |
|               |   TableScan: test projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment], partial_filters=[test.l_orderkey > Int64(0)]                                                                                |
| physical_plan | CoalesceBatchesExec: target_batch_size=8192                                                                                |
|               |   FilterExec: l_orderkey@0 > 0                                                                                |
|               |     ParquetExec: file_groups={4 groups: [[Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:0..10165445], [Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:10165445..20330890], [Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:20330890..30496335], [Users/pmcgleen/work/arrow-datafusion/datafusion-cli/part-0.parquet:30496335..40661778]]}, projection=[l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment], predicate=l_orderkey@0 > 0, pruning_predicate=l_orderkey_max@0 > 0, required_guarantees=[] |
|               |                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.012 seconds.

@alamb  this looks ok to me (unless I've missed something).    file_groups = 4 means it's loaded in parallel on each of the 4 CPUs available?

@alamb  this looks ok to me (unless I've missed something).    file_groups = 4 means it's loaded in parallel on each of the 4 CPUs available?

I agree -- thank you for checking @pmcgleenon . Let's close this issue and we can open new issues for future improvements if warranted


Release plan?

Hi! I'm looking forward this library and I want to know which kind of release plan do you have for a stable version, any idea? I think some explanation on wiki or readme could help.
Thanks!

We do not have a defined release plan as of now. When users report issues / give pull requests, I fix them. If it is a substantial fix, I release new version, else, I wait for a good amount of smaller fixes to accumulate before giving out a release.


[flutter_tools] "Resource temporarily unavilable" running "/usr/bin/xcrun simctl list"

Stacktrace on stable/2.5.1:
Thread 0 main threadProcessException: ProcessException: Resource temporarily unavailable  Command: /usr/bin/xcrun simctl list --json devices
at _ProcessImpl._start(process_patch.dart:390)
at Process.start(process_patch.dart:36)
at _runNonInteractiveProcess(process_patch.dart:565)
at Process.run(process_patch.dart:47)
at LocalProcessManager.run(local_process_manager.dart:74)
at ErrorHandlingProcessManager.run.<anonymous closure>(error_handling_io.dart:669)
at _run(error_handling_io.dart:567)
at ErrorHandlingProcessManager.run(error_handling_io.dart:668)
at _DefaultProcessUtils.run(process.dart:267)
at SimControl._list(simulators.dart:130)
at SimControl.getDevices(simulators.dart:155)
at SimControl.getConnectedDevices(simulators.dart:171)
at IOSSimulatorUtils.getAttachedDevices(simulators.dart:76)
at IOSSimulators.pollingGetDevices(simulators.dart:50)
at PollingDeviceDiscovery._initTimer.<anonymous closure>(device.dart:386)
at PollingDeviceDiscovery._initTimer.<anonymous closure>(device.dart:384)
at _rootRun(zone.dart:1420)
at _CustomZone.run(zone.dart:1328)
at _CustomZone.runGuarded(zone.dart:1236)
at _CustomZone.bindCallbackGuarded.<anonymous closure>(zone.dart:1276)
at _rootRun(zone.dart:1428)
at _CustomZone.run(zone.dart:1328)
at _CustomZone.bindCallback.<anonymous closure>(zone.dart:1260)
at Timer._createTimer.<anonymous closure>(timer_patch.dart:18)
at _Timer._runTimers(timer_impl.dart:395)
at _RawReceivePortImpl._handleMessage(isolate_patch.dart:184)

cc @jmagman
Maybe this is related to hitting the ulimit of maxprocs? https://apple.stackexchange.com/questions/373035/fix-fork-resource-temporarily-unavailable-on-os-x-macos

Maybe? I've seen that as "Too many open files, errno =24" https://github.com/flutter/flutter/issues/2976. We could print a message suggesting they up their ulimit (and put in ~/.zshrc or wherever based on $SHELL) for either case.


Fix wrappers build issues

Fix make rbthemis_uninstall

Checklist

[ ] Change is covered by automated tests
[ ] Benchmark results are attached (if applicable)
[ ] The coding guidelines are followed
[ ] Public API has proper documentation
[ ] Example projects and code samples are up-to-date (in case of API changes)
[ ] Changelog is updated (in case of notable or breaking changes)

please, add one more step to ruby tests - https://github.com/cossacklabs/themis/blob/master/.github/workflows/test-ruby.yaml
let's check that uninstallation works properly too

i wonder why tests are failing


Crash when going deeper into Bleak Falls Barrow

Hi, first of all, thank you for this wonderful mod list! Until now I've had a wonderful time.
But after going deeper into Black Falls Barrow the game crashes every time. It looks to me that some kind of npc is stumbling in a trap which causes the game to crash.
I've tried so far to increase the page file to 20GB (my pc has 32gb ram) and to check all files for integrity. But nothing helped.
Please have a look at the attached log:
Crash_2024_6_23_19-20-28.txt

After leaving the dungeon and entering again, everything works. Unfortunately I had crashes in some other random locations.


release: setup logging when ensuring operator installed

Towards https://github.com/giantswarm/giantswarm/issues/4355.

@MarcelMue PTALA


If the Packagel can identity pure text?

This Package is fantastic, however, it just provides with 20 classes without non-programming language(Pure Text). Can Further version add the function

Hi @LiuHao-THU ,
I think that a "Pure Text" class would be too broad to fit in the current model, as pure text can be written in any language with any alphabet and contain any type of characters.
However, to reduce the chances of misclassifying pure text I added a check on the prediction probabilities.
That said, some text files may still be categorized as Markdown (as Markdown is a text formatting language) and Guesslang may have issues classifying some short code snippets, as stated in the documentation.
Thank you for the feedback.


tmplock must be closed before removal

defer are called in LIFO style, so os.Remove was called before tmplock.Close (which is obviously wrong).
Since this a dependency, I don't know how it should be fixed...
(don't update to the latest version of https://github.com/nightlyone/lockfile: if the same process acquire a lock two times, it won't fail because the pid is the same)

Great find! This could explain #108 where this temporary file is not removed because there is still an open file handler.

(don't update to the latest version of https://github.com/nightlyone/lockfile: if the same process acquire a lock two times, it won't fail because the pid is the same)

I am aware of this situation (see https://github.com/nightlyone/lockfile/pull/15#issuecomment-248988619) and that's why I forked the repository to revert this change. For now we are using my personal version of it which is located at https://github.com/Acconut/lockfile/tree/v1.0.0. If you could send this PR to this repository, I would be very thankful!

https://github.com/Acconut/lockfile/tree/v1.0.0. If you could send this PR to this repository, I would be very thankful!

But I can't make a PR on a tag (and your master is up to date with https://github.com/nightlyone/lockfile)

That's correct, please still open a PR against the master branch and I will take care of changing the behaviour of the master branch.

The dependency with the necessary changes has been updated in 7f26757a40b27166b773aec306e63533e4db6747. Thank you for noticing :)


The secondary page can't respond to any GestureDetector

The secondary page can't respond to any GestureDetector.
Is it not supported GestureDetector ？ or is there a problem with my code？

Related to (https://github.com/xuelongqy/flutter_easy_refresh/pull/714/commits/f7e12a65a9f3c85f753edbfdb6df9f910bd2555c)

3.3.1+2 fixed