It's not working

Hello!
I'm trying to use this plugin, here's my gulp task:
gulp.task('webserver', function() {
    gulp.src('./build/')
        .pipe(webserver({
            directoryListing: true
        }))
    ;
});

When I run this task and open URL in the browser I'm getting list of files from the root of the project, not from the build subdirectory. When I try to navigate to the /build/index.html it show me: Cannot GET /build/index.html.
Please advise. Thanks!

gulp --version
[gulp] CLI version 3.6.2
[gulp] Local version 3.8.10

Update
gulp-connect is working fine with the following code:

gulp.task('webserver', function() {
    connect.server({
        root: './build',
        port: 8888,
        fallback: 'index.html'
    });
});

I wasn't being able to find any working Node web server to use with my single page application, so I've created a simple node server of my own on top of Connect and server-static. Maybe it will help someone: https://github.com/betsol/spa-server. Cheers!

@slavafomin
make sure you are using a return statement in _all your streams, I experienced this same thing and going through all my gulp tasks and making sure they returned fix this issue for me

@slavafomin I would start with disabling the directoryListing option. Either by removing, as false by default, or by setting as directoryListing: false. My setup is working here is the config I am using.
// gulpfile.js
var gulp = require('gulp');
var webserver = require('gulp-webserver');

gulp.task('webserver', function() {
  gulp.src('public')
  .pipe(webserver({
    livereload: true,
    directoryListing: false,
    open: true
  }));
});

gulp.task('default', ['webserver']);

output from tree:
├── gulpfile.js
├── index.html
├── package.json
└── public
    ├── css
    │   └── style.css
    ├── img
    ├── index.html
    └── js

:+1: @alnjxn


fix filtering bug in filtering unnest cols and dim cols: Received a non-applicable rewrite

Fixes #XXXX.

Description
This PR fixes the bug in the following query, currently it throws the exception org.apache.druid.query.QueryException: Received a non-applicable rewrite: {j0.unnest=c2}, filter's dimension: c1
Whenever we have And / OR filters on unnest columns, we rewrite the filters on unnest cols.
We were covering OR filters and disqualifying the rewrite for non unnest cols and we have to to do same thing with AND filters as well.  This PR checks for Boolean filter and disqualify the rewrite for non unnest cols.
Adding the 3 tests that covers the given scenario.
select * from (
          select * from (
                         select * from mytest, unnest(mv_to_array(c2)) as u(n1)
                     ), unnest(mv_to_array(c2)) as u(n2)
               ), unnest(mv_to_array(c2)) as u(n3)
where n1='A' and c1>=1

3rd test case cover the nested filer scenario for unnest filters, This PR address the unpack the nested OR/AND filters and attempt to rewrite only for qualified filters.
SELECT dimZipf, dim3_unnest1, dim3_unnest2, dim3_unnest3 FROM
      ( SELECT * FROM
           ( SELECT * FROM lotsocolumns, UNNEST(MV_TO_ARRAY(dimMultivalEnumerated)) as ut(dim3_unnest1) )           ,UNNEST(MV_TO_ARRAY(dimMultivalEnumerated)) as ut(dim3_unnest2)
      ), UNNEST(MV_TO_ARRAY(dimMultivalEnumerated)) as ut(dim3_unnest3)  WHERE dimZipf=27 AND (dim3_unnest1='Baz' OR dim3_unnest2='Hello') AND dim3_unnest3='World'

Thank you

Fixed the bug ...
Renamed the class ...
Added a forbidden-apis entry ...

Release note

Key changed/added classes in this PR

MyFoo
OurBar
TheirBaz

This PR has:

[x] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[x] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[x] added integration tests.
[x] been tested in a test Druid cluster.

There are 2 cases:

Consider the query ```
select * from foo, UNNEST(dim3) as u(d3) where d3 IN (a,b) or m1 < 10

Here since d3 can be mapped directly to an input column dim3, the filter can be reconstructed as a pre-filter with dim3 IN (a,b) or m1<10 and pushed to the base cursor.

2. Consider the query

select * from foo, UNNEST(ARRAY[dim1,dim2]) as u(d12) where d12 IN (a,b) or m1 < 10
 since the filter outside contains an OR filter and `d12` has no direct match to an input column there should be no pre-filters and the entire filter should be just used as a post-filter. In the current code, this needs to be accounted for and a test case needs to be added for the same. Currently, for the failing query in CI, this does not seem to be the case and returns incorrect results

filterSplitter.filtersForPostUnnestCursor = {ArrayList@10853}  size = 1
0 = {OrFilter@8617} "(m1 as numeric < 2 || j0.unnest IN (a, aa))"
filterSplitter.filtersPushedDownToBaseCursor = {ArrayList@10854}  size = 1
0 = {OrFilter@10852} "(m1 as numeric < 2)"


Define the tasks necessary to update the infrastructure software architecture

Sub-tasks:

Flowchart creation of the order of operations for activation/deactivation for the instances
Identify the different architecture pieces that will be affected (networking, SSL certs, other instances)
Create a testing document for planning on what tests will need to be completed
Schedule meetings with other stakeholders and have them contribute to the testing document with an understanding that they are responsible for handling their own testing.

Stakeholders: IEC PSC, IEC Pitt, Harvard


Geospatial Support

One of the reasons I like the frictionlessdata JSON Table Schema is that is supports a portable spatial type, geojson (also in topojson format). I'm in municipal government and a lot of our data is geospatial.
My organization specifically uses Postgres and Oracle for geospatial. Postgres/Postgis is very popular for geospatial. Ideally for these database types, we would create a geometry field and convert the geojson to a geometry. GeoAlchemy adds spatial support SQLAlchemy, but only supports Postgis.
I'm willing to contribute, but need guidance as to whether it makes sense to include geospatial in this lib/repo, and if so, how best to.

@awm33
Hi. Great! Let me prepare in coming days some ideas for how it could be added to the library. So you'll be able to contribute.

@roll Awesome!
I started playing around here https://github.com/CityOfPhiladelphia/jsontableschema-sql-py/tree/geometry
I used geoalchemy with some changes to it's geometry type to use geojson to pull fields in and out https://github.com/CityOfPhiladelphia/jsontableschema-sql-py/blob/geometry/jsontableschema_sql/mappers.py#L15
I'm also taking a crack at SDE using a similar method. I'm calling it OracleSDE but I think this would work for postgres and orace SDE. Think of SDE as similar to postgis, it adds geospatial types and functions to the database.

@awm33
Sorry for the late response I've been on holidays.
I'd like to ask for ideas here from @akariv and @amercader. Adam was working pretty closely on usage of jsontableschem-sql-py and Adria is a geospatial specialist.


Blog Search tutorial broken?

Tried to get started with the Blog Search Tutorial but the Kaggle dataset no longer seems to be available. I logged into Kaggle, accepted their terms, got a key, and tried downloading using the kaggle api client. Also tried downloading from the site. If the data is no longer available, perhaps make a copy of the data available elsewhere or use a different dataset.

Just tested this and the data is still available. You need to be logged on, and after that the direct link https://www.kaggle.com/c/predict-wordpress-likes/download/trainPosts.zip should work. I haven't tested their api client.
Alternatively, you can try to go to https://www.kaggle.com/c/predict-wordpress-likes/data, and click on the trainPosts.zip and use the download button there.

Thanks for the response lesters. I'll try downloading again. Maybe the kaggle site had a bad day when I tried the download.

I tried again, this time with Chrome in addition to Firefox. I go to the page where I accept the license, click the button, but it doesn't do anything. Disabled my popup blocker; no luck.
I understand this is not Vespa's problem, but posting the follow up here to alert you to potential problem. Your links may be working if you already had access, but appears new requests are no longer going through. I strongly urge you to use data that you control and own rather than rely on a third party who may or may not understand how their dataset is being used.

Sorry to hear your problems with Kaggle. Did you try logging in through a Google or Facebook or Yahoo account? Anyway, there is a small sample data file in the repository that might unblock you: https://github.com/vespa-engine/sample-apps/tree/master/blog-tutorial-shared

I used my Google account to sign in. I can see and download other data. In any case I'll prepare data that I intend to evaluate Vespa with, so this data set won't block me.
Thank you for your follow-up.


feat(certificate): db enablement company certificates

Description
Enable companies to fetch their company certificates with the respective document status.
Why
To ensure that the feature can get supported, first changes on db level are needed ++ seeding data adjustments. Tables added are mentioned below
CompanyCertificate
CompanyCertificateStatus
CompanyCertificateType
CompanyCertificateTypeDescription
Issue
#471
Checklist
Please delete options that are not relevant.

[x] I have followed the contributing guidelines
[x] I have performed a self-review of my own code
[x] I have successfully tested my changes locally
[x] I have added tests that prove my changes work
[x] I have checked that new and existing tests pass locally with my changes
[x] I have commented my code, particularly in hard-to-understand areas

@jjeroch the implementation in this PR adds a Id field to the company_certificates table. That differentiate to the description in the issue and I couldn't find a discussion in the issue nor Jira. Main reason for that is if we don't have the field a company could only have one entry linked to company_certificate_type. Example BMW has an ISO 9001certificate which is inactive, they could never create a new one, only way to have an ISO 9001 certificate is to update the inactive one, in my opinion we shouldn't do that, do you agree with that or should Anurag remove the Id field?
@AnuragNagpure one general thing I recognised, please use conventional commits not only for the PR title. Please use it for your commit messages as well

Hi @Phil91  & @AnuragNagpure  fully agree with your proposal. BMW must have the chance to have an ISO 9001certificate in status "INACTIVE" while a second is in status "ACTIVE"


Added html style

Added markup with HTML
Added Styles with CSS
Added Linters
Configured LInters
Added a descriptive readme file
Added license.

Worked on Errors shown by Linters.
Linters are working perfectly fine


Implement -then -else for Ternary operator

Considering the ternary operator was added using the syntax most friendly to c# developers and not general PowerShell users - i.e all your admin users,
I propose that the ternary operator should be further extended to also allow -then -else syntax
My reason for asking is simple - there are still many in the PowerShell community that want & prefer the verboseness of the language over other languages and find that the verboseness provides a much easier readability experience.
PowerShell is known for allowing users to pick syntax options that they are comfortable and giving -then -else syntax would help bring the ternary operator to those users, that don't come from the classical developer type roles.

Referencing this comment from @lzybkr https://github.com/PowerShell/PowerShell/issues/3239#issuecomment-365072448
The syntax that I believe is most PowerShelly would be $x -then $y -else $z & is the syntax that I am requesting also be implemented to aid in possible adoption of this operator

It's worth mentioning as it kinda feels like it's been forgotten that PowerShell's primary target userbase was always Windows Administrators and NOT C# Developers, whether Windows based or not.
I don't think that it's too much of an ask to implement this before v7 goes GA to appease both camps of users

It was already discussed here but the reaction was quite negative: https://twitter.com/Steve_MSFT/status/1152013489965109252

If you want a bit more verboseness what is wrong with just using the existing if statement if ($x) {$y} else {$z}?  I honestly don't see much advantage of adding -then / -else operators.  It saves two chars:
$x -then $y -else $z
if ($x) {$y} else {$z}

And only one char if your code style is:
$x -then $y -else $z
if($x) {$y} else {$z}

Character count isn't everything. Typing parentheses and braces is slightly awkward, and honestly it's hard to read on if/then/else on one line. Of course the ternary operator can be hard to read too, which is why my proposed syntax doesn't seem universally hated.

That's true, as well. My only concern for using -then/-else syntax for a ternary operator is that it creates an exception in the language rules.
All other operators with -operator syntax are purely binary operators. While they may have multiple arguments on their LHS or RHS, they only have a LHS and RHS argument. None of them have a third argument.
From a newbie's perspective, $a -then $b -else $c looks pretty similar to $a -eq $b -as $c (or any other consecutive sequence of operators; they're not super common, but also not unheard of). This creates a pitfall in terms of the expectation of what an -operator does and how it should work. The ternary syntax is a completely different mode of operation than all existing operators using this syntax.
The expectation we currently have for an -operator is:

$LHS -operator $RHS # creates some definite result that can be further operated on

Extending this to the proposed syntax creates a false assumption:

$a -then $b and $a -else $b are also valid syntaxes, and $a -then $b -else $c is just an extended variant of these.

I'm not saying this is a hard-line "we should under no circumstances do this", but I would think that given the proposal is to create, effectively, a "clearer" version of the ternary syntax, we should probably avoid creating a behaviour that is easily misunderstood based on currently reasonable assumptions about how the language patterns behave.

That said, there's a fairly easy solution here: implement -then and -else as separate operators, which can then be combined to create this ternary syntax.
For example:
$a -then $b # if $a is truthy, returns $b, otherwise returns $false or $null

$a -else $b # if $a is truthy, returns $a, otherwise returns $b

$a -then $b -else $c # if a is truthy, returns $b; otherwise the first operation is evaluated, leaving us with: $false -else $c, which then returns $c

I'm not especially fond of -then/-else as being implemented in precisely the same way as the current ternary operator. I don't like introducing exceptions to established and presently well-defined language rules if we can help it.

Someone in the PowerShell Discord brought up one potential issue with my above idea:
$true -then $null -else $someValue

If implemented as separate operators... this could be a potential pitfall if it is not handled carefully by the parser. Ternary syntaxes should return $null in that kind of case ($true ? $null : $value will return $null) but if the operators are completely independent it's possible that -else could be mistakenly triggered.
I still think that's the best way to proceed, but we'd need to add a special case somewhere in the logic to also skip an immediately following -else if the -then evaluates to a non-null value.

I'm just reading this issue for the first time now.

TL;DR: I'm in favor of it as long as $a -then $b and $a -else $c don't create parse errors, essentially.

Yeah, that's how && and || work in JavaScript. I think that's the most logical way for them to operate. Essentially the same as -and and -or, but returning the value rather than the boolean coercion of that value.
I don't see any particular composition of that as a pitfall, just a regular result of left-associative operator parsing.
I think the value of these operators is in being able to pass out a meaningful result in a truthy situation, but stop an error in a falsey situation:
return $item -then $item.Transform()

I personally think this is still needed from an accessibility perspective for ease of scan readability for those that may be new to the language / new to writing any code and even to those that know exactly what they are looking at with the current operator.
I realise that I seem on this point be in the minority but as more and more people pick up coding and languages like PowerShell we should realistically keep accessibility and how readability by tooling like screen readers will play an important part in future.
I also would like to see a new feature to tooling like PSScriptAnalyzer and an accessibility scanning report generator in future, which I know would be a huge undertaking to achieve.
Keeping this open for now and will discuss this further in future.

I personally don't see any benefit to adding a THIRD way of writing the same thing. Plus, I am quite worried about adding new operators since PowerShell already has some arguably messed up precedence - 1 -or 1 -and 0 or 1, 2 + 3, 4. How should something like $a ? $b : $c -then $d -else $e be parsed? ($a ? $b : $c) -then $d -else $e or $a ? $b : ($c -then $d -else $e)? The only way I could see solving it is by making -then and -else aliases for ? and : respectively, though that opens up questions whether mixing the two with $x ? $y -else $z and $x -then $y -else $z should be allowed - probably not.

I'd still like to see this for accessibility reasons but would say that it's priority is lower than other issues but would like it if a PR for this could come from either the team or the community in the future.


Dense versus Sparse Models

Could you please explain this sentence 'To assess the performance and efficiency benefits of sparse architectures in time series forecasting, we replaced the MoE layer with a dense layer containing an equivalent number of parameters as the activated parameters in the MoE layer.' in the paper?Can I explain the activated parameters as the parameters of two experts chosen by each token from the eight experts? If the number of parameters is controlled in this way, the number of parameters in both models should be the same, right? Why is Figure 3 comparing Time-moe base and dense200MB, and Time-moe large and dense1B specifically? The number of parameters in these two kinds of models is different. Thanks for your reply!

Thank you for your interest in our work!
You are correct. The number of activated parameters for the dense and MoE models is the same across the base, large, and ultra. For instance, the Dense_base model has 50 million parameters, and the TimeMoE_base model has 50 million activated parameters, although it has a total of 113 parameters.
On the right side of Figure 3, the comparison between the red and blue bars depicts the performance gap between different-scale Dense and Time-MoE models.
On the left side of Fig. 3, we compare TimeMoE_base with Dense_200M and TimeMoE_large with Dense_1B. The purpose of this comparison is to demonstrate that the TimeMoE model outperforms the Dense model in forecasting, even when it has fewer activated parameters. This implies lower inference costs, lower training costs, yet high performance. If the comparison is made under the condition of the same number of activated parameters, it merely showcases the performance, which is already illustrated on the right side of Figure 3.
Please feel free to contact us, if you have any questions.


Load animal treatment

Summary Change Description

add new example file containing animal treatments source from Michael Neinast
serum_lactate_timecourse_treatment header and data tweaks
tests now load serum_lactate_timecourse_treatment
tests tweaked to account for additional samples, animals, and study
utils SampleTableLoader now casts tracer_labeled_count as an int
utils SampleTableLoader now casts time_collected as a float
added eol last line of serum_lactate_timecourse_treatment.tsv
removed default category from Protocol model
moved retrieve_protocol method logic from Accucor loader to a class method in Protocol
added treatment to obob_sample_table_headers.yaml
sample loader now creates treatment protocols, if column is defined

Affected Issue Numbers

Resolves #94

Code Review Notes
Describe any areas of concern that code reviewers should pay particular
attention to.  E.g. There are significant logic changes in function X.
Checklist

[ ] All issue requirements satisfied (or no linked issues)
[x] Linting passes.
[x] Migrations created & committed (or no model changes)
[x] Tests implemented (or no code changes)
[x] All tests pass

Given no checkmark in the checklist for the issue requirements, are there specific requirements in the issue that weren't implemented that I can keep in mind during review?

Given no checkmark in the checklist for the issue requirements, are there specific requirements in the issue that weren't implemented that I can keep in mind during review?

I believe the first 4 bullet points were implemented in this PR.
The last 2 (test-related) were implemented by @lparsons in an earlier PR, I believe.
Possibly the only thing  missing would be "deleting a protocol while an animal is associated with it" test?


adapter-netlify missing dependency in output function

Recently, I've been getting this weird error with adapter-netlify:
12:59:10 PM:   Error message
12:59:10 PM:   A Netlify Function is using "encoding" but that dependency has not been installed yet.
12:59:10 PM:
12:59:10 PM:   By default, dependencies inside a Netlify Function's "package.json" are not automatically installed.
  There are several ways to fix this problem:
12:59:10 PM:     - Removing your Function's "package.json" and adding the dependencies to the project's top-level "package.json" instead. This is the fastest and safest solution.
12:59:10 PM:     - Running "npm install" or "yarn" inside your Netlify Function in your build command.
12:59:10 PM:     - Adding the following plugin to your "netlify.toml":
12:59:10 PM:
12:59:10 PM:   [[plugins]]
12:59:10 PM:   package = "@netlify/plugin-functions-install-core"

Adding the suggested plugin doesn't seem to help...
I have not been able to reproduce the issue in a minimal repo, but my repo is public and this commit reproduces the issue.

I just tried deploying the latest SvelteKit demo to Netlify without issue:

https://sveltekit-demo-netlify-2021-05-04.netlify.app/
https://github.com/Leftium/sveltekit-demo-netlify

So I don't think this is a SvelteKit issue, but rather an issue with your project.

Normally using adapter-netlify@next is recommended, but what you really want is the version that matches your SvelteKit template project. Because SvelteKit is in beta, there is no guarantee that the template project from Apr 3rd will be compatible with an adapter from Apr 30th, or even Apr 13th.
You may be able to fix the adapter issues in your project by installing a version of the adapter from around the date you initialized the project: https://www.npmjs.com/package/@sveltejs/adapter-netlify?activeTab=versions
However, what I would do is:

Start a new SvelteKit project with the latest template
Get the netlify adapter working
Apply your project's changes on top of this new project.

I have basically already done steps 1 and 2, so you can just clone/fork my repo:
https://github.com/Leftium/sveltekit-demo-netlify

I can't run your demo. It just says supabaseUrl is required. But you should put next as the version for both SvelteKit and the adapter in your package.json and then delete your package-lock.json and run npm install again.
I'm going to close this since it's working in the default project and I can't reproduce

I'm experiencing the exact same problem but my error code is:
A Netlify Function is using "exports" but that dependency has not been installed yet.
Tried to install 'exports' but got the following error:
npm ERR! notsup Unsupported engine for<EMAIL_ADDRESS>wanted: {"node":">= 0.6.0 < 0.7.0"} (current: {"node":"14.17.0","npm":"6.14.13"})
FWIW if anyone is interested I'd greatly appreciate help with this, I'm totally flummoxed.


Geology Plate is missing link to the source dataset

This plate intentionally has no legend because there are so many categories. For other plates, links to the source data are shown adjacent to the legend / table output. We should add the link to the source dataset near the query result and/or at the bottom of the map, even before any query is made.

This ticket can be closed; we're not showing a Geology plate anymore -- we just link out to the USGS map.


Question about overlapping / colinear lines

Hi,
i am using your implementation to write a trim-line tool for Inkscape. Basically this already works really fine except that i am not able to handle colinear line segments (lines with same slope) which are overlapping each other. At the moment there are no intersections returned for such cases like in example:

Do you have some tip for me where to fork in adjusting the code properly to return intersection points, trimmed segments or merged segments?
https://stackoverflow.com/questions/48348228/overlapping-segments-in-bentley-ottmann-algorithm
best regards, Mario

I don't think this can be supported efficiently as part of the sweep-line stepping logic as it would require, checking that any of the segments that share the same range on the scan-line overlap, this could potentially be a very expensive operation, that loses the benefits gained by using the sweep-line in the 1st place.
While I haven't looked into this thoroughly, I think the most straightforward solution to this if to pre-process the line segments to remove overlap.

sort segments by their gradient.

i fixed it by implementing another algorithm, found at https://gist.github.com/sbma44/dc34e5005d9827aa7b1c8c11e68b0c6b


TensorFlow 2+Keras support

I'm trying to use this code with TensorFlow 2.2.0 (latest) and Keras 2.3.0 (built into tf) and I'm running into some errors.
import tensorflow as tf
import receptive_field as rf

g = tf.Graph()
with g.as_default():
    model = tf.keras.applications.InceptionV3()
    output_name = "inception_v3/conv2d_85/Conv2D"

    images = tf.Variable(tf.ones(shape=(1, 299, 299, 3)), name="input_image")
    model(images)

    rf_x, rf_y, eff_stride_x, eff_stride_y, eff_pad_x, eff_pad_y = \
      rf.compute_receptive_field_from_graph_def( \
      g.as_graph_def(), "input_image", output_name)

(tf.placeholder from the sample code replaced with tf.Variable)
Output:
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-48-2edcc65a3292> in <module>
      9     rf_x, rf_y, eff_stride_x, eff_stride_y, eff_pad_x, eff_pad_y = \
     10       rf.compute_receptive_field_from_graph_def( \
---> 11       g.as_graph_def(), "input_image", output_name)

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/receptive_field/python/util/receptive_field_computation.py in compute_receptive_field_from_graph_def(graph_def, input_node, output_node, stop_propagation, input_resolution)
    272       (kernel_size_x, kernel_size_y, stride_x, stride_y, padding_x, padding_y,
    273        _, _) = parse_layer_parameters.get_layer_params(
--> 274            node, name_to_node, node_info[node.name].input_size)
    275       logging.vlog(
    276           3, "kernel_size_x = %s, kernel_size_y = %s, "

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/receptive_field/python/util/parse_layer_parameters.py in get_layer_params(node, name_to_node, input_resolution, force)
    292   if node.op == "Conv2D" or node.op == "DepthwiseConv2dNative":
    293     stride_x, stride_y = _stride_size(node, name_to_node)
--> 294     kernel_size_x, kernel_size_y = _conv_kernel_size(node, name_to_node)
    295     # Compute the padding for this node separately for each direction.
    296     total_padding_x, padding_x = _padding_size_conv_pool(

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/receptive_field/python/util/parse_layer_parameters.py in _conv_kernel_size(node, name_to_node)
     95         "Weight layer's name input to conv layer does not end with '/read' or "
     96         "'/Conv2D/ReadVariableOp': %s" % weights_layer_read_name)
---> 97   weights_node = name_to_node[weights_layer_param_name]
     98   if weights_node.op == "VariableV2" or weights_node.op == "VarHandleOp":
     99     shape_dim = weights_node.attr["shape"].shape.dim

KeyError: 'inception_v3/conv2d_85/kernel'

Similarly for VGG:
g = tf.Graph()
with g.as_default():
    model = tf.keras.applications.VGG19()
    output_name = "vgg19/block5_conv4/Conv2D"

    images = tf.Variable(tf.ones(shape=(1, 224, 224, 3)), name="input_image")
    model(images)

    rf_x, rf_y, eff_stride_x, eff_stride_y, eff_pad_x, eff_pad_y = \
      rf.compute_receptive_field_from_graph_def( \
      g.as_graph_def(), "input_image", output_name)

~/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/receptive_field/python/util/parse_layer_parameters.py in _conv_kernel_size(node, name_to_node)
     95         "Weight layer's name input to conv layer does not end with '/read' or "
     96         "'/Conv2D/ReadVariableOp': %s" % weights_layer_read_name)
---> 97   weights_node = name_to_node[weights_layer_param_name]
     98   if weights_node.op == "VariableV2" or weights_node.op == "VarHandleOp":
     99     shape_dim = weights_node.attr["shape"].shape.dim

KeyError: 'vgg19/block5_conv4/kernel'

Could you please let me know what's going wrong and how to fix it?
I tried with TF 1.15.2 and Keras 2.2.4-tf and got a different set of errors, so I don't think downgrading is enough to fix this.
It would be really nice if this worked with the latest set of models (eg from tensorflow hub or https://github.com/tensorflow/models/), and Keras is now officially in TF as well so it would be nice if it was supported out of the box.

Hi,
If you'd like to get a  response from the authors, take a look at here:

Sorry for the extremely long delay! Indeed I had not been notified of this issue. In any case, here goes the answer -- using this library with tf.keras.applications models works :)
Here's how to compute it for the InceptionV3 example you tried, @aizvorski
import tensorflow.compat.v1 as tf
import receptive_field as rf

g = tf.Graph()
with g.as_default():
  tf.keras.backend.set_learning_phase(0) # Disable BN learning.
  x = tf.keras.Input([None, None, 3], name='input_image')
  model = tf.keras.applications.InceptionV3(input_tensor=x)

graph_def = g.as_graph_def()

input_node = 'input_image'
output_node = 'conv2d_85/Conv2D'
(receptive_field_x, receptive_field_y, effective_stride_x,
 effective_stride_y, effective_padding_x, effective_padding_y) = (
         rf.compute_receptive_field_from_graph_def(graph_def, input_node,
                                                   output_node))

# These will print 1183.
print(receptive_field_x)
print(receptive_field_y)

# These will print 32.
print(effective_stride_x)
print(effective_stride_y)

# These will print 554.
print(effective_padding_x)
print(effective_padding_y)

This gives identical numbers as the Mixed_7b layer from the Slim inception_v3 model (see table).


waterline-sql-builder error on criteria value 'NOT', 'AND', 'OR', 'IN', 'NOTIN'

Sails version: 1.1.0
Node version: v10.1.0
NPM version: 6.7.0
DB adapter name: sails-mysql
DB adapter version: 1.0.1
Operating system: docker node image

https://github.com/max918/waterline-sql-builder-error-on-criteria-value-NOT-AND-OR-IN-NOTIN-
simple test in config/bootstrap.js

AdapterError: Unexpected error from database adapter: Undefined binding(s) detected when compiling SELECT query: select `createdAt`, `updatedAt`, `id`, `code`, `order` from `area` where `code` in (?) and `order` = ? limit ?

@max918 - Hi, thanks for bringing attention to this. Do you have an example with further details?

@max918 - Hi, thanks for bringing attention to this. Do you have an example with further details?

@johnabrams7
here a model for testing ORM method.
`/**

Area.js

@description :: A model definition represents a database table/collection.
@docs        :: https://sailsjs.com/docs/concepts/models-and-orm/models
*/

module.exports = {
attributes: {
//  ╔═╗╦═╗╦╔╦╗╦╔╦╗╦╦  ╦╔═╗╔═╗
//  ╠═╝╠╦╝║║║║║ ║ ║╚╗╔╝║╣ ╚═╗
//  ╩  ╩╚═╩╩ ╩╩ ╩ ╩ ╚╝ ╚═╝╚═╝

code: {
  type: 'string',
  columnType: 'char(2)',
  unique: true
},

order: {
  type: 'number',
  defaultsTo: 0
}

//  ╔═╗╔╦╗╔╗ ╔═╗╔╦╗╔═╗
//  ║╣ ║║║╠╩╗║╣  ║║╚═╗
//  ╚═╝╩ ╩╚═╝╚═╝═╩╝╚═╝

//  ╔═╗╔═╗╔═╗╔═╗╔═╗╦╔═╗╔╦╗╦╔═╗╔╗╔╔═╗
//  ╠═╣╚═╗╚═╗║ ║║  ║╠═╣ ║ ║║ ║║║║╚═╗
//  ╩ ╩╚═╝╚═╝╚═╝╚═╝╩╩ ╩ ╩ ╩╚═╝╝╚╝╚═╝

},
};
`
testing on bootstrap.js
`/**

Seed Function
(sails.config.bootstrap)

A function that runs just before your Sails app gets lifted.

Need more flexibility?  You can also create a hook.

For more information on seeding your app with fake data, check out:
https://sailsjs.com/config/bootstrap
*/

module.exports.bootstrap = async function() {
// single criteria, no problem
await Area.find({
code: 'IN'
})
// multi criteria, BUG
await Area.find({
code: 'IN',
order: 0
})
// By convention, this is a good place to set up fake data during development.
//
// For example:
//   // // Set up fake development data (or if we already have some, avast)   // if (await User.count() > 0) {   //   return;   // }   //   // await User.createEach([   //   { emailAddress<EMAIL_ADDRESS>fullName: 'Ryan Dahl', },   //   { emailAddress<EMAIL_ADDRESS>fullName: 'Rachael Shaw', },   //   // etc.   // ]);   //
};
`
if there are only single WHERE case, no error.
but if more than one WHERE case and the value include the "special keyword", throw that AdapterError.
reproducible test case is provided on
https://github.com/max918/waterline-sql-builder-error-on-criteria-value-NOT-AND-OR-IN-NOTIN-
thanks.

@max918 Thanks for the example info and repo 👍  Feel free to submit a PR for this one.  In the meantime, we are currently developing an upcoming sails-sql adapter as well.

@max918 I cloned your repo and was able to reproduce the error. This definitely looks like a bug, and it specifically seems to be from the all-caps 'IN'. (If I changed code to any other value, there were no errors, but even if I changed the attribute name and did a query e.g. foo: 'IN', I saw the error - and only when multiple criteria were used.)
If you are using MySQL, you can switch to the new sails-sql adapter in place of sails-mysql (as @johnabrams7 mentioned above): I tested the example repo locally with sails-sql and didn't see the error anymore. Hope that helps!


Fix captions ui bug

What
Fix captions ui bug
Why
https://skype.visualstudio.com/SPOOL/_workitems/edit/3217733
https://skype.visualstudio.com/SPOOL/_workitems/edit/3217735
https://skype.visualstudio.com/SPOOL/_workitems/edit/3217737
https://skype.visualstudio.com/SPOOL/_workitems/edit/3217740
How Tested
Tested on samples

Process & policy checklist

[ ] I have updated the project documentation to reflect my changes if necessary.
[ ] I have read the CONTRIBUTING documentation.

Is this a breaking change?

[ ] This change causes current functionality to break.

Caption settings still shows double scroll bar

Updates, pushed a fix for this double scroll bar problem

NICE!


Typing mismatch on provider & observer on ver ^2.2.4

Versions

vee-validate: 2.2.4-5

Describe the bug
I get on latest version.
TS2749 'ValidationObserver' refers to a value, but is being used as a type here.
It works fine if you install exactly 2.2.3

The default exports were changed to fix #2026 which, The main issue was exporting a "Component Definition" is different than exporting a "Vue Component" which caused some issues in projects.
You can use "ValidationProviderInstance" and "ValidationObserverInstance" types instead when using them as cast targets for $refs, which I assume you are currently doing.
Feel free to provide the snippet where it fails so I can help more.

Another related problem is that these can't be used in TSX files. Making them classes, as opposed to interfaces, fixes the issue.
export class ValidationProviderInstance extends Vue { ... }
and
export class ValidationObserverInstance extends Vue { ... }


Fresh installation via Snap on Ubuntu 22.04 Remote (Nomachine) fails to start.

Lapce Version
0.2.4 ( Actual Ubuntu snap Version)
System information
Ubuntu 22.04
Describe the bug
Cant start Lapse on my remote Ubuntu after fresh installation wth snap
Additional information
$ lapce -w (lapce:113225): Gtk-WARNING **: 10:55:22.460: cannot open display: :1001
$ sudo /etc/NX/nxserver --list
NX> 127 Available sessions:

Display Username Remote IP       Session ID                       Node
------- -------- --------------- -------------------------------- --------------
1001    dt       <IP_ADDRESS>       5CEED9666B4B90BC829B2536AC5BCF9E localhost:4000
1002    dt       xxx.xxx.xxx.xxx 2DB06A4BF9C0B62F0AE734356D5E1EFD localhost:4000

$ echo $XDG_CURRENT_DESKTOP
ubuntu:GNOME

I remember running into the issue with Firefox as well, and if I remember correctly, this NoMachine KB article helped me make it work https://kb.nomachine.com/AR06T01163
I don't think it's a Lapce issue, more like NoMachine + Snap integrations.
There are also other instances,

https://kb.nomachine.com/TR11T10698
This one indicates a recent change in snap may have rebroken NoMachine: https://kb.nomachine.com/TR03V11118

Snap is not supported. Use prebuilt binary or one of the .deb packages available on Releases page


Make etcdmain/serve.go public and more modular

I'd like to discuss making etcdmain/serve.go structs and methods public and more modular for easier use by third parties that are embedded the etcd server. This might also involve refactoring / changing part of startEtcd(). I'm willing to take a crack at the patch if it's something that the project maintainers would be interested in having.
Let me know, thanks!

It would be interesting... But we do not have bandwidth to do this now. If you have any thoughts or plan about how to do it. We are happy to listen!

@xiang90 I will try and write the patch assuming you like the idea. If there are any gotchas or issues you foresee I'd appreciate if you could warn me.
Thanks!

@purpleidea
Start() should probably have several stages, like read user conf -> optional discovery -> prepare etcd server conf -> start http/grpc server -> start etcd server. etc..
One thing we want to do is to move all discovery related logic into one func.
Ideally, user can do Discovery(), which hints the user about how to start the server and where are other peers. There is some discovery logic here inside etcdserver pkg:https://github.com/coreos/etcd/blob/master/etcdserver/server.go#L301 that should be moved to etcdmain.

I've taken a first crack at this in: https://github.com/coreos/etcd/pull/5432
I tested this on v.3.0.0-beta.0, but rebased it to git master to share. Please let me know if anything particular or behaviour wise would have changed since then that would affect this.
Thanks!

bump this to 3.1

It's probably worth mentioning that https://github.com/coreos/etcd/pull/5584 is my recent update to this.

Other potential users of an improved embedded etcd users:

bootkube https://github.com/coreos/bootkube/issues/31 cc @aaronlevy
localkube https://github.com/redspread/localkube @ethernetdan
hyperkube https://github.com/kubernetes/features/issues/11 @jbeda


Angular 2 + ImmutableJS causes lot of rerenders

Printing out log in ngOnChanges of a component having inputs bound with @select immutables. It prints out a lot of times even when the app is idle.

I don't think this is an issue with immutable-js, rather with the way your app is using angular.

Closing this issue since there's isn't actionable information here. If you can provide a reproducible issue, then please feel free to reopen the issue.


[VL] Update get_velox.sh for macos

What changes were proposed in this pull request?
fix velox_ep/scripts/setup-macos.sh
folly needs to be installed after install_double_conversion
https://github.com/oap-project/velox/blob/1910a1722dcb5e7dde83e84defdc33f21327be84/scripts/setup-macos.sh#L116-L124
How was this patch tested?
manual tests

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_3293_time.csv
log/native_master_09_26_2023_dffd4a993_time.csv
difference
percentage

q1
43.04
43.11
0.069
100.16%

q2
25.36
24.77
-0.590
97.67%

q3
37.52
37.78
0.259
100.69%

q4
41.58
41.79
0.209
100.50%

q5
71.02
70.23
-0.787
98.89%

q6
6.65
6.52
-0.136
97.95%

q7
86.77
86.01
-0.764
99.12%

q8
80.34
79.93
-0.412
99.49%

q9
116.19
115.79
-0.391
99.66%

q10
47.42
46.51
-0.914
98.07%

q11
19.59
19.54
-0.047
99.76%

q12
23.26
24.04
0.777
103.34%

q13
49.11
50.28
1.172
102.39%

q14
16.30
14.27
-2.034
87.52%

q15
28.40
27.99
-0.403
98.58%

q16
15.62
15.97
0.347
102.22%

q17
121.92
120.66
-1.263
98.96%

q18
163.31
163.74
0.429
100.26%

q19
13.41
12.08
-1.328
90.09%

q20
28.79
27.57
-1.219
95.77%

q21
237.05
239.19
2.144
100.90%

q22
15.41
15.34
-0.070
99.54%

total
1288.07
1283.12
-4.952
99.62%


More Darknet Mirrors (Tor, Freenet, ZeroNet)

Hi Guys,
we already have a I2P mirror. Does someone have experience with Tor, Freenet and ZeroNet? It would be nice to mirror this GitHub repo in more darknets.
Help wanted! 👍

Note: fork for darkwebs

I can try to take care of this.

Tor mirror set up.
I'm generating onion addresses.
So far:
ptoolsg6yhcioili.onion (current)
ptoolsdl67nwcg2u.onion

@beardog108 has experience with ZeroNet.

Yep, i'm getting a ZeroNet mirror setup. Will post here & on the subreddit when Its up.
I will need people to help me seed.

Currently generating a vanity address for the ZeroNet mirror.

Alright, so i've got a fork up.
I couldn't get a vanity address to work for it, because for some reason it would not list the site in my client. I could probably get it to work by fooling around with it more, but i don't really feel like it.
You can access the site (assuming i'm still seeding) with your ZeroNet client & a browser here: http://<IP_ADDRESS>:43110/14gSEpfWfF5Go7u5y7UidZQCnGa6eSBYZ9/

Next we should do Freenet & IPFS would be good too.

Corrected the ZeroNet site 'home' button, as it would take you to your ZeroNet router rather than the top of the page.

Hi guys, thank you both so much. I've added links on the website, subreddit and posted it to the zeronet subreddit already: https://www.reddit.com/r/zeronet/comments/5lwomw/privacytoolsio_is_now_available_on_zeronet_please/

If you want a freenet mirror you'll need a stripped down version that doesn't rely on JS.

Which along with "no one uses Freenet" is an argument not to support Freenet :P

Hey guys, I have one suggestion to make this lower friction for your users.
By default I2P does not know the mapping from privacytools.i2p -> wfewiqkdt5bvowfp3igmkaksjvcvwj7bag2smqpqyu5x4hges2qa.b32.i2p . If you make the link on your site:
" http://privacytools.i2p/?i2paddresshelper=?i2paddresshelper=wfewiqkdt5bvowfp3igmkaksjvcvwj7bag2smqpqyu5x4hges2qa.b32.i2p "
then a user won't have to go through the I2P jump servers. That special URL will tell the I2P router the mapping quicker.

Hi, the title of the ZeroNet mirror can be changed via the sidebar. Drag the 0 button to the left to see the sidebar.
Zcash.png was missing. It might be forgotten. Try adding this file and sign the site again.
The site is running out of size limit (10MB). You can define optional files so that these files will not be counted when ZeroNet is adding up file sizes. Don't worry, once an optional file is requested by browser, ZeroNet will download that file immediately. Open content.json with a text editor. After the modified key, add an optional key with a regular expression string bound to it.
"modified":<PHONE_NUMBER>.648398,
"optional": "(fonts/.*)",

Save all of your changes and sign the site.

The limit can be increased and Zcash.png is missing on privacytools.io too.

guys pleasee give me hacking sites on tor

@beardog108 Are you still maintaining the Zeronet mirror? It seems to default to /classic, so I think it's a bit outdated.

Since we now have an actual server, @JonahAragon and @BurungHantu1605 could set up all these mirrors?

Probably yeah. We have Tor up, what other ones do we need? I’m not super familiar with darknets so I’m not sure what’s “in” right now.

I2P and ZeroNet, in my opinion. Both are extremely simple to set up. I suggest using i2pd for hosting the I2P service. It's written in C, so it has a much lower overhead than the "official" Java I2P. Also you don't need all the Java I2P features to host a service.

From darknets I agree on I2P (even if I am not sure on I2Pd as I am having https://github.com/PurpleI2P/i2pd/issues/1275), I guess ZeroNet is currently "in" and I would also like to see Yggdrasil (https://github.com/privacytoolsIO/privacytools.io/issues/685) even if it's more of a graynet and IPFS (https://github.com/privacytoolsIO/privacytools.io/issues/139#issuecomment-270243582) even if it currently has no darknet support.

Zeronet: is not made to be anonymous protocol as someone might imagine similar to hidden services:

https://zeronet.readthedocs.io/en/latest/faq/#is-zeronet-anonymous

Freenet: not well studied/matured like Tor and I2P, its decentralized data store (not newbies friendly) because it needs you to maintian storage capacity ,  its still poses danger of the copied unknown data when downloaded in the user space like malicious PDF or docs or so which freenet cant protect the user, written in java = SLOWNESS , You need to design the web to work with freenet because for e.g it doesnt allow JS ....

I2P-Java: Not good for Eepsites since it eats alot of resources , its still on the old days of monolithic design where everything put together whether you need it or not thats why you will find email client, built-in webserver (jetty) , Console comes with JS ...etc not good at least for eepsites.

I2Pd: Written in C++ , lighter than I2Pj , and comes as a router only. (needs Terminal/UI configurations in order to switch your preferences), Suitable for Eepsites.

I think we should now make the decision : do we want to spend time and resources on mirrors on lesser used and known anonymity networks.

I would like to publish some static website on to .onion, freenet, zeronet and why not some other. Is there some tools to help doing this from my CI ?
PS : where should i ask this in a better place than here ?


WFLY-6576 mod_cluster advertise="false" does not disable advertise if…

… there are no proxies configured
Jira
https://issues.jboss.org/browse/WFLY-6576

Looks good to me. @pferraro please approve.

@jfclere Any problem with this "functionality" going away?

@pferraro @jfclere do either of you have anything to add?

That looks good for me. @pferraro any comments?

:+1:


Resources without relations

This PR prevents relations trying to be created when a resource doesn't have any relations.
Currently if you try to create a resource that doesn't have any relationships, it will crash
Uncaught exception! TypeError: Object.keys called on non-object

[x] This seems legit
[ ] This will work!

Look legit to me. It probably makes sense to add some unit test coverage for these function, but it's out of scope for now, since we're already using this fix successfully in the dependent projects, so let's get it merged and released: :+1: :shipit:

:+1:


Issue with beautification of JavaScript in an inline XML-like template

The issue born from discussion at https://github.com/Glavin001/atom-beautify/issues/755.
It seems that there are issues when beautify JavaScript code inline a XML file (the JavaScript is used inside an application server template file).
See the original ticket for code examples.

Thanks for the heads up, I've commented there.

As discussed at https://github.com/Glavin001/atom-beautify/issues/755 the issue disappeared and I can't give context to reproduce it anymore.


Inner join with select

I am having a hard time achieving this scenario
https://www.w3schools.com/sql/trysql.asp?filename=trysql_select_join_inner
SQL For selecting the lowest OrderDetailId for an OrderId
SELECT Orders.OrderID, OrderDetails.OrderDetailID
FROM Orders
INNER JOIN (
    SELECT *
    FROM OrderDetails
    ORDER BY OrderDetailID ASC
) OrderDetails ON Orders.OrderId = OrderDetails.OrderId
WHERE Orders.OrderID == '10248'
group by Orders.OrderID;

SQL For selecting the lowest OrderDetailId for an OrderId
SELECT Orders.OrderID, OrderDetails.OrderDetailID
FROM Orders
INNER JOIN (
    SELECT *
    FROM OrderDetails
    ORDER BY OrderDetailID DESC
) OrderDetails ON Orders.OrderId = OrderDetails.OrderId
WHERE Orders.OrderID == '10248'
group by Orders.OrderID;

But I am failing to get the same result with MOOR
    return (select(dbOrderTable).join([
      innerJoin(
        dbOrderDetailTable,
        dbOrderDetailTable.orderId.equalsExp(dbOrderTable.orderId),
      )
    ])
          ..orderBy([
            OrderingTerm.desc(dbOrderTable.orderDetailId),
          ])
          ..groupBy([dbOrderTable.orderId]))
          .get();

After 2 hours I figured out how to do it with a custom query but a custom query is not the preferred way
    final orderTableName = dbOrderTable.$tableName;
    final orderDetailTableName = dbOrderDetailTable.$tableName;
    final sqlQuery = '''
    SELECT *
    FROM $orderTableName
    INNER JOIN (
      SELECT *
      FROM $orderDetailTableName
      ORDER BY ${dbOrderDetailTable.orderDetailId.$name} DESC
    ) $orderDetailTableName ON $orderTableName.${dbOrderTable.orderId.$name} = $orderDetailTableName.${dbOrderDetailTable.orderId.$name}
    GROUP BY $orderTableName.${dbOrderTable.orderId.$name};
    ''';
    return customSelect(sqlQuery).get();

I would like to se an api for inner join selects

For us the compile time checked option is not possible because we need to do the mapping ourselfs.
I have created an extra ticket for tracking that. #615

Subqueries with the Dart API would make the query builder even more powerful. I can see them very useful not ony with joins, but as one value expressions to use as extra columns for a select statement.
SELECT table.*,
    (SELECT something from other_table WHERE ...)
FROM table

The moor files can be used for subqueries, but the Dart API can be greatly benefitial for complex parametrized queries where the  select and where statements change based on the input.


[QUESTION] I was able to download the LLM in llama.cpp, but I cannot download the LLM in Transformers.

I want to run 70B on llama-3-instruct using a GPU, so I am trying to run 8B on a GPU first.
I have confirmed that there is already a bug in the gguf model using llama.cpp that prevents the use of GPUs.
So I wanted to use pytorch's model with Transfoemer, but I can't download it.
I think this may have already been reported, or it may be a configuration error on my part, so I would appreciate it if you could help me.
settings

error
user@398d4e0508fb:/workspace# xinference launch --model-engine llama.cpp --model-name llama-3-instruct --size-in-billions 8 --model-format ggufv2 --quantization Q4_K_M --gpu-idx 0,1
Launch model name: llama-3-instruct with kwargs: {}
Model uid: llama-3-instruct-VxQZAWuj
user@398d4e0508fb:/workspace# xinference launch --model-engine Transformers --model-name llama-3-instruct --size-in-billions 8 --model-format pytorch --quantization 4-bit --gpu-idx 0,1
Launch model name: llama-3-instruct with kwargs: {}
Traceback (most recent call last):
  File "/opt/conda/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 898, in model_launch
    model_uid = client.launch_model(
  File "/opt/conda/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 911, in launch_model
    raise RuntimeError(
RuntimeError: Failed to launch model, detail: [address=<IP_ADDRESS>:50729, pid=43] Failed to download model 'llama-3-instruct' (size: 8, format: pytorch) after multiple retries

Python 3.10.13
xinference v.0.12.2.post1

Python 3.10.13 xinference v.0.12.2.post1

贴一下后端的报错，不仅仅是命令行报错

OK!
Is this enough? If not, please let us know.
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-667ea709-40b704097bad97526456dd7c;c8a0107e-2241-49ff-9f19-817625902912)

Cannot access gated repo for url https://huggingface.co/api/models/meta-llama/Meta-Llama-3-8B-Instruct/revision/main.
Access to model meta-llama/Meta-Llama-3-8B-Instruct is restricted. You must be authenticated to access it.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/xinference/api/restful_api.py", line 771, in launch_model
    model_uid = await (await self._get_supervisor_ref()).launch_builtin_model(
  File "/opt/conda/lib/python3.10/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/opt/conda/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/opt/conda/lib/python3.10/site-packages/xoscar/backends/pool.py", line 659, in send
    result = await self._run_coro(message.message_id, coro)
  File "/opt/conda/lib/python3.10/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/opt/conda/lib/python3.10/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/opt/conda/lib/python3.10/site-packages/xinference/core/supervisor.py", line 837, in launch_builtin_model
    await _launch_model()
  File "/opt/conda/lib/python3.10/site-packages/xinference/core/supervisor.py", line 801, in _launch_model
    await _launch_one_model(rep_model_uid)
  File "/opt/conda/lib/python3.10/site-packages/xinference/core/supervisor.py", line 782, in _launch_one_model
    await worker_ref.launch_builtin_model(
  File "xoscar/core.pyx", line 284, in __pyx_actor_method_wrapper
    async with lock:
  File "xoscar/core.pyx", line 287, in xoscar.core.__pyx_actor_method_wrapper
    result = await result
  File "/opt/conda/lib/python3.10/site-packages/xinference/core/utils.py", line 45, in wrapped
    ret = await func(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/xinference/core/worker.py", line 641, in launch_builtin_model
    model, model_description = await asyncio.to_thread(
  File "/opt/conda/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/opt/conda/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/conda/lib/python3.10/site-packages/xinference/model/core.py", line 67, in create_model_instance
    return create_llm_model_instance(
  File "/opt/conda/lib/python3.10/site-packages/xinference/model/llm/core.py", line 223, in create_llm_model_instance
    save_path = cache(llm_family, llm_spec, quantization)
  File "/opt/conda/lib/python3.10/site-packages/xinference/model/llm/llm_family.py", line 297, in cache
    return cache_from_huggingface(llm_family, llm_spec, quantization)
  File "/opt/conda/lib/python3.10/site-packages/xinference/model/llm/llm_family.py", line 824, in cache_from_huggingface
    download_dir = retry_download(
  File "/opt/conda/lib/python3.10/site-packages/xinference/model/utils.py", line 127, in retry_download
    raise RuntimeError(
RuntimeError: [address=<IP_ADDRESS>:28136, pid=42] Failed to download model 'llama-3-instruct' (size: 8, format: pytorch) after multiple retries

huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-667ea709-40b704097bad97526456dd7c;c8a0107e-2241-49ff-9f19-817625902912)
Cannot access gated repo for url https://huggingface.co/api/models/meta-llama/Meta-Llama-3-8B-Instruct/revision/main.
Access to model meta-llama/Meta-Llama-3-8B-Instruct is restricted. You must be authenticated to access it.

huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-667ea709-40b704097bad97526456dd7c;c8a0107e-2241-49ff-9f19-817625902912)

Cannot access gated repo for url https://huggingface.co/api/models/meta-llama/Meta-Llama-3-8B-Instruct/revision/main.
Access to model meta-llama/Meta-Llama-3-8B-Instruct is restricted. You must be authenticated to access it.

Llama-3 is a kind of model that needs to be authenticated. Refer to this: https://inference.readthedocs.io/en/latest/getting_started/troubleshooting.html to set the environment variable.

Thank you for your answer.


Last character from optional message is omitted

Message in this transaction
"test send"
Message in the wallet
"test sen"

I included this bug fix in frontend-v2 branch.


Loss curve anomalies

Thank you for sharing your work. May I ask why the loss of the validation dataset has been rising? this doesn't look normal.

Can you run his cloab demo? Why I run a lot of errors, the most obvious error is"ImportError: Using the Trainer with PyTorch requires accelerate>=0.20.1: Please run pip install transformers[torch] or `pip install accelerate "I tried everything, but nothing worked


Funds that I transferred have disappeared..

So I was transferring money out of my Nami account to a different account, I checked to make sure many times I was sending it to the right address. So I go to send the money and it went through but now the money is stuck on the block chain and I can access the funds pls me out I have screenshots etc to prove it

Please submit a support request here: https://iohk.zendesk.com/hc/en-us/requests/new


Adding framer Motion to Home Page

Description
The website will look more appealing and attractive after adding framer motion
Kindly assign this issue to me under hactoberfest, gssoc-ext and relevant tags.

thank you for raising the issue, but currently the animations have been already implemented in the home page.
if you can enhance it in a better way, you can go ahead, if so, kindly explain what would you enhance. ;)


Kubernetes: Use AKS curated OS images by default

What this PR does / why we need it: Uses AKS-curated OS images by default.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
If applicable:

[x] documentation
[ ] unit tests
[x] tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

Kubernetes: Use AKS curated OS images by default

/hold
will update image version to 0.10.0 once it's published

Is there any documentation around what this new Distribution contains? Is this a fork of Ubuntu or an entirely new Distribution? Will this apply to acs-engine nodes as well, or only AKS?

@neurot1cal not yet, we are still in testing phase. The image is based on Ubuntu 16.04-LTS  (same as previous default) and contains pre-installed software needed for deployment that was previously fetched at provisioning time. The objective is to reduce deployment time (significantly) and eliminate transient errors due to general download flakiness. The plan is to enable this for both acs-engine Linux nodes (at first) and AKS nodes (after baking in acs-engine for a bit).

/lgtm

/lgtm

@neurot1cal this is in master if you want to give it a try :)

@ablekh this is already on in AKS worldwide as of mid-October. There are still more improvements coming as this is a continual process to our commitment to deliver fast, reliable deployments. Unfortunately AKS cluster deletion is not affected by this change and not related to the acs-engine project which only does cluster provisioning. AKS handles on the service side (some improvements might be coming there too).

@CecileRobertMichon Thank you so much for your update. Hmm ... then, based on the time frame that you mentioned, unfortunately, it means that this change was definitely not nearly enough to get cluster creation speed down to feasible numbers (please see details in my issue referenced above). Having said that, I definitely appreciate the team's commitment and hard work. I hope that the goal of achieving fast and reliable AKS deployments will be reached sooner, rather than later. :-) And, for that, perhaps, some changes beyond acs-engine are needed. Re: cluster deletion - Understood. I hope that relevant team(s) at MS will address this issue as well.

@ablekh out of curiosity what range of durations are you seeing in AKS deployments lately? Have you noticed any improvements recently? Agreed that more work is needed to get there, this effort was just one part of it. After this change, we've observed average cluster deployments in acs-engine drop to ~4 minutes for 3 nodes and ~7 minutes for 55 nodes. AKS create is built on top off acs-engine but there are more operations that happen in AKS provisioning than in acs-engine so it needs additional time.

@CecileRobertMichon Well, the number that I shared in my issue 711 referenced above, unfortunately, apply largely to end of October - beginning of November time frame. Hence, my original comment here. Almost always, I create a 3-node cluster on AKS (and then scale as needed), so my numbers refer to that size. Based on my recollection, an average time for creating a 3-node AKS cluster that I experienced was ~20 min. IMO, 80% of the total time spent on other (non-acs-engine) operations seems like way too much to me ...

@CecileRobertMichon You're very welcome. And thank you very much, too. :-)


[Feature] Add luau syntax in code block

Before Requesting

[X] I found no existing issue matching my feature request

Describe the feature you'd like!
Since Luau (superset of Lua) is expanding, I'd like to see that BetterDiscord support their syntax in codeblock.
Anything else?
No response

Do you mean the codeblocks as in the the container messages use for code?
This is an example codeblock.

Codeblock syntax highlighting is handled by Discord itself. A plugin could extend it but it's not something BD itself will handle.

I’ve seen Vencord able to use luau syntax in their codeblock though, that’s why I asked here if BetterDiscord able to do it too

Are you sure that wasn't just stock discord behavior? Because I'm pretty sure vencord doesn't touch codeblocks either. I know they have a plugin to enhance them but that plugin does not add extended language support.


Feature/better fill last line

I noticed that it still wasn't quite getting as many notations on the last line of the score as it could, and this pull request fixes that by ignoring the trailingSpace on the last notation

Regarding the trailingSpace: it makes more sense to me to simply set the last notation of the score to have a trailingSpace of 0; this could be done once when the notations are created in the Gabc code. This would avoid extra logic in the layout phase and eliminate some math operations from the layout loop too.

I tried doing that, but then it needs to get updated when calling updateMappingsFromSource(), so I thought it made more sense to handle it when laying out the line, since that way none of the layout would need to be updated if additional GABC code is added at the end, causing a notation to no longer be the last in the score.

Ah, true. OK, looks good as is, thx.


Broken with last CBA update

The login process is now broken. Hopefully it's just the login to fix, and the rest of the functionality still works!

Hi, I'm currently maintaining a Python API for CommBank (not publically released yet). I've just updated it to support the new login process, so hopefully, I can save you guys some time 👍 .
The login process is similar to before except before you are redirected to the homepage (and are logged in) you now need to manually submit a hidden form in the response, then you are taken to the homepage.
CommBank has added an API for accounts now 😀 /retail/netbank/api/home/v1/accounts this returns a JSON object with a list of the accounts, their balances etc. and a link to their transactions page. The parsing for the transactions page has stayed the same, except when navigating to it for the first time you are required to submit a hidden form in the response then you will then be redirected to the transactions page.
The addition of the hidden forms is due to CommBank moving over to OpenID.

Thanks for the info Josh!
On Tue, 17 Dec 2019 at 12:19 am Josh Willox<EMAIL_ADDRESS>wrote:

Hi, I'm currently maintaining a Python API for CommBank (not publically
released yet). I've just updated it to support the new login process, so
hopefully, I can save you guys some time 👍 .
The login process is similar to before except before you are redirected to
the homepage (and are logged in) you now need to manually submit a hidden
form in the response, then you are taken to the homepage.
CommBank has added an API for accounts now 😀
/retail/netbank/api/home/v1/accounts this returns a JSON object with a
list of the accounts, their balances etc. and a link to their transactions
page. The parsing for the transactions page has stayed the same, except
when navigating to it for the first time you are required to submit a
hidden form in the response then you will then be redirected to the
transactions page.
The addition of the hidden forms is due to CommBank moving over to OpenID.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/twang2218/node-cba-netbank/issues/17?email_source=notifications&email_token=AAE2X43COM2CY6YUKPZB2PDQY5573A5CNFSM4J2MCG72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG6VTKI#issuecomment-566057385,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE2X4ZJZIB5MYDVQ7XDV33QY5573ANCNFSM4J2MCG7Q
.

A little update, CommBank appears to have reverted the login process back to normal, so everything should work as it did before. Along with that, the API endpoint for accounts is no longer available 😢, so guess its back to web scraping 👍.

Yep, my new code started failing on 2019-12-31 at 4:55 PM
On Sat, 4 Jan 2020 at 00:38, Josh Willox<EMAIL_ADDRESS>wrote:

A little update, CommBank appears to have reverted the login process back
to normal, so everything should work as it did before. Along with that, the
API endpoint for accounts is no longer available 😢, so guess its back to
web scraping 👍.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/twang2218/node-cba-netbank/issues/17?email_source=notifications&email_token=AAE2X43QU2XWP55WO2X5ZODQ345XFA5CNFSM4J2MCG72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIBEMAI#issuecomment-570574337,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE2X43UZTHV4L5YAF3QWFDQ345XFANCNFSM4J2MCG7Q
.

I feel like I should update this thread with the new changes, commbank has re-enabled that API I was talking about, which means you need to handle those hidden forms I mentioned previously.
Accounts API
https://www.commbank.com.au/retail/netbank/api/home/v1/accounts

Sample python code to extract the relevant data.
[
    {
        "name": account["displayName"],
        "bsb": account["number"][:6],  # first 6 digits are the bsb.
        "number": account["number"][6:],  # rest is the account number.
        "balance": account["balance"][0]["amount"],
        "available_balance": account["availableFunds"][0]["amount"],
        "link": account["link"]["url"],
    }
    for account in response.json()["accounts"]
]

I was trying out mitmproxy the other day and was able to glean some of commbank's mobile APIs. Here's what I found:

all requests need the header Authorization: Bearer <token>.
not completely sure where to get this token but I believe you can extract it from the web login process.

all responses from the APIs are JSON formatted.

Get Balances (i.e get accounts)
https://www.commbank.com.au/innovate/SimpleBalance/v1/balance

Get Transactions
Needs the header x-param-map-accountIdentifier: <product_code>+<bsb><account_number>
https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions

Get Transaction Detail
Each transaction from the ../transactions endpoint has an id field this needs to be used to get the detailed information for a transaction.
https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions/accountIdentifier-<id>/details

Awesome, thanks :)

Hey @jcwillox is your python wrapper ready for release by any chance? 🤞

@svict4 haha kinda, the code base isn't great as I wrote it a long time ago, but it does work. I spent a bit refactoring just now to make it a little more like jcwillox/up-bank-api. I'll upload the code to GitHub when I get the chance. I don't use commbank that much anymore so it's probably not going to be maintained that well, and currently, it only supports retrieving accounts and the first page of transactions, but maybe someone can improve upon it.

Fair enough; Up is the better bank 💪
Was simply looking for some prior art, so thanks for your previous comment on the various endpoints

Maybe https://www.commbank.com.au/Developer/ will be of use, however there is no docs on how to authenticate with the apis

It would be great if we could use those but unfortunately, you can't unless you're a business.

To access consumer APIs, you'll need to be accredited by the ACCC and get the customer's consent

For now we are stuck, mimicking the NetBank login process and then using the following unofficial API endpoints.
https://www.commbank.com.au/retail/netbank/accounts/api/accounts
https://www.commbank.com.au/retail/netbank/accounts/api/transactions

I’ve swapped mostly to headless chrome. I use puppeteer to login, then I do
a few api calls in the context of the page to get JSON
On Thu, 28 Jul 2022 at 7:10 pm Benjamin Lupton @.***>
wrote:

Maybe https://www.commbank.com.au/Developer/ will be of use, however
there is no docs on how to authenticate with the apis
—
Reply to this email directly, view it on GitHub
https://github.com/twang2218/node-cba-netbank/issues/17#issuecomment-1197876478,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE2X42TE3IPB3QO2TBZUVTVWJFATANCNFSM4J2MCG7Q
.
You are receiving this because you authored the thread.Message ID:
@.***>

I feel like I should update this thread with the new changes, commbank has re-enabled that API I was talking about, which means you need to handle those hidden forms I mentioned previously.
Accounts API
https://www.commbank.com.au/retail/netbank/api/home/v1/accounts

Sample python code to extract the relevant data.
[
    {
        "name": account["displayName"],
        "bsb": account["number"][:6],  # first 6 digits are the bsb.
        "number": account["number"][6:],  # rest is the account number.
        "balance": account["balance"][0]["amount"],
        "available_balance": account["availableFunds"][0]["amount"],
        "link": account["link"]["url"],
    }
    for account in response.json()["accounts"]
]

I was trying out mitmproxy the other day and was able to glean some of commbank's mobile APIs. Here's what I found:

all requests need the header Authorization: Bearer <token>.
not completely sure where to get this token but I believe you can extract it from the web login process.

all responses from the APIs are JSON formatted.

Get Balances (i.e get accounts)
https://www.commbank.com.au/innovate/SimpleBalance/v1/balance

Get Transactions
Needs the header x-param-map-accountIdentifier: <product_code>+<bsb><account_number>
https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions

Get Transaction Detail
Each transaction from the ../transactions endpoint has an id field this needs to be used to get the detailed information for a transaction.
https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions/accountIdentifier-<id>/details

@jcwillox Did you ever work out how to find that token during the login phase? I am scraping in php, and could only see XSRF-TOKEN, but seems not to work?

@Paytah232 no sorry never looked into those mobile APIs any further. I just pushed the CommBank API wrapper I wrote in Python all those years ago so maybe that will help you, I believe it still works. It sounds like you're having issues with the login so maybe check out this function https://github.com/jcwillox/commbank-api/blob/9ed0f4d1107c28bddb45cc07d6d3be97cbf82178/commbank/client.py#L26-L69.
Here's the repo with the Python API client, it definitely needs some polish, but it might be helpful to people https://github.com/jcwillox/commbank-api.

@jcwillox Thanks Josh. Seems the hardest thing I'm finding with PHP implementation is scraping the crawler for a form that allows me to post data. From what I can tell, both of these scripts are creating a form and submitting it, but I don't think PHP can do that - at least, I haven't found a way yet, and I haven't found an existing form I can use once I get past login.
https://github.com/jcwillox/commbank-api/blob/0364bfccc03c3c1ff9d70e6529f6908e44bc0f40/commbank/client.py#L51-L56
https://github.com/twang2218/node-cba-netbank/blob/7feb68bf8d2580a4f04c75c0c00bb2308a5aa31c/src/api.js#L164-L176
I can however get the 40 most recent transactions using the transactions api. I'm wondering if you found a way to pass a query or header to that api that allowed it to handle to/from dates, increase results, or get more results. I wouldn't even know where to start to try that myself.
I have a forked repo where I'm slowly working on this for a PHP implementation here:
https://github.com/paytah232/php-cba-netbank

@paytah232 seems this got lost in all my github notifications just cleaning them out now.
The code you linked in my repo, does a post request to login which returns HTML, then I'm using an html-parser (BeautifulSoup) to grab the action which is the next URL and find all the inputs and pull out their names and values. Basically the browser would auto submit this form but because we're headless we have to manually pull out the form data and send the request ourselves, (parse_form is just a helper to handle extracting any HTML forms data). Then finally in the section you highlighted I submit that form by sending its data to the action url.
This should definitely be possible in PHP I'm sure they'll have some kind of HTML parse library available as that's the only thing you're missing.

@jcwillox No worries mate.
What html are you actually receiving from client.py?
I could be interpreting incorrectly, but you're using your action to parse the html that is received after the successful login. So, that should be the page listing your account information and you are finding any form, grabbing the action/data, and then creating a new form to return.
For whatever reason, I don't believe I am actually ever getting to this page where I can find a form to get data from. The original source I don't think finds it's form (#aspnetForm) because it doesn't return any accounts and there is no form on the page. I end up skipping that and using the api after login because it was the only way I could get any data. So I am definitely logged in, but perhaps not correctly redirecting to the account page.
I also note that when I do try to redirect to the account page like this, I always get an error stating that the url doesn't exist. I am guessing that the 'dynamic'  account link I received is now not valid (or has changed) for that session.
I also have seen a response page that asks for a 'Click to Confirm' prompt, which your wrapper doesn't cover. I'm wondering if there's a reason I'm getting that too...
Perhaps (offline) we could compare notes on stages of parsing in more detail?

Looking at your code, the crawler you use looks to be either more advanced or actually controlling a headless browser, in my case, I have to manually parse those forms and construct the correct requests for each step of the login, but your crawler appears to actually allow you to click a button which is cool, so you may not need to do the same process.
I've never tried redirecting to the accounts page so not sure why that wouldn't work but that sounds quite possible. But for getting the accounts/transactions the JSON API is definitely the way to go.
As for getting transactions, I use the link in the response from the accounts JSON endpoint, e.g. accounts[0].link.url, to create the URL for the transactions JSON endpoint, I do that here https://github.com/jcwillox/commbank-api/blob/main/commbank/client.py#L113-L115 e.g.
accounts[0].link.url='/retail/netbank/accounts/?account=<some-long-id>'
transactions URL='https://www.commbank.com.au/retail/netbank/accounts/api/transactions?account=<some-long-id>'

Here's an example of how the parse_form works during the login
So the first parse_form gets passed a huge HTML document, with an HTML form inside it, here's part of it.

   <title>
      NetBank - Log on to NetBank - Enjoy simple and secure online banking from Commonwealth Bank
   </title>
   <meta name="description" content="NetBank is here to simplify your banking life. You can manage all your accounts
      from one place, and do your banking whenever or wherever it suits you." />
   <meta name="google-site-verification"
      content="_Y1ecy6XcbQ3abYLk9glqe_Csuq0QakknnlXfW2Qrjo" />
   <link rel="canonical"
      href="https://www.my.commbank.com.au/netbank/Logon/Logon.aspx" />
   <meta name="viewport" content="width=device-width,
      initial-scale=1" />
   <link rel="stylesheet" type="text/css"
      href="https://static.my.commbank.com.au/static/netbank/theme/fo/css/logon-merge.8397238ab0ae7a25ea1af4d375f2c3df.css"
      rel-album="R700" />

   <form method="post" action="/netbank/Logon/Logon.aspx" onsubmit="javascript:return WebForm_OnSubmit();" id="form1"autocomplete="off">

      <input type="hidden" name="RID" id="RID" value="..." />
      <input type="hidden" name="SID" id="SID" value="..." />
      <input type="hidden" name="cid" id="cid" value="..." />
      <input type="hidden" name="rqid" id="rqid" value="..." />
      <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE"
         value="..." />

This gets parsed into the following, I redacted the values as I can't remember if any of these are private. The actual response
{
    'action': '/netbank/Logon/Logon.aspx',
    'data': {
        'RID': '...',
        'SID': '...',
        'cid': '...',
        'rqid': '...',
        '__VIEWSTATE': '...',
        '__VIEWSTATEGENERATOR': '...',
        '__EVENTVALIDATION': '...',
        'JS': 'D'
    }
}

The second parse_form gets

      <meta http-equiv='X-UA-Compatible' content='IE=edge' />
      <base target='_self'/>

      <form method='post' action='https://www.commbank.com.au/retail/netbank/identity/signin-oidc'>
         <input type='hidden' name='code' value='<redacted>' />
         <input type='hidden' name='scope' value='openid profile digital-platform netbank' />
         <input type='hidden' name='state' value='<redacted>' />
         <input type='hidden' name='session_state' value='<redacted>' />
         <button>Click to continue</button>
      </form>

Which is parsed to
{
    'action': 'https://www.commbank.com.au/retail/netbank/identity/signin-oidc',
    'data': {
        'code': '<redacted>',
        'scope': 'openid profile digital-platform netbank',
        'state': '<redacted>',
        'session_state': '<redacted>'
    }
}

Perhaps (offline) we could compare notes on stages of parsing in more detail?

Sure 👍

I’ve switched to headless chrome do minimal parsing
I only fill out login and click buttons, and here’s the trick, I add chrome driver network listener and just wait for the JSON I want to come back during the click-around session

Yeah that sounds like a pretty solid method 👍. At least the CommBank login process that I've been using has been stable for a few years now, so I haven't had to do much maintenance.

Just as a random comment, I just don't think I have a good enough understanding of the crawler for PHP, so I haven't made any progress.
I did however find a 3rd party app that is relatively easily integratable for PHP, Python and Java - called Odoo.
It's free (for one app - I used the Accounting app), and provides API access to a BUNCH of banking institutions through other 3rd party providers, all claiming best in practice security (hopefully true).
I've got it set up and working for my PHP DB, and I'm pretty happy with it overall. Anyway, thanks for the help whilst I was trying to go down the crawler path!

Worth noting that cloudflare workers no support headless browsing, so that can be an avenue for achieving this.
I'm also wondering if anyone has any experience with getting transactions from a Zip Pay account. I'm thinking of going headless, or using a proxy to observe the API requests.

CF workers are V8 isolates (js runtime only) whereas lambda is a
firecracker vm (Linux machine)
On Wed, 31 May 2023 at 5:53 pm Benjamin Lupton @.***>
wrote:

Worth noting that cloudflare workers no support headless browsing, so that
can be an avenue for achieving this.
I'm also wondering if anyone has any experience with getting transactions
from a Zip Pay account. I'm thinking of going headless, or using a proxy to
observe the API requests.
—
Reply to this email directly, view it on GitHub
https://github.com/twang2218/node-cba-netbank/issues/17#issuecomment-1569677268,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE2X425ET5V7WKPE75YN7DXI32HTANCNFSM4J2MCG7Q
.
You are receiving this because you authored the thread.Message ID:
@.***>

In hindsight, it actually makes sense for bank to not provide a public API, as poorly implemented commercial software using these API's would be a major security risk.
Xero accounting software is ACCC accredited, and supports a wide range of Australian banks for automated bank feeds.
1. ANZ (Australia and New Zealand Banking Group)
2. Commonwealth Bank of Australia (CommBank)
3. National Australia Bank (NAB)
4. Westpac Banking Corporation
5. Bank of Queensland (BOQ)
6. Bendigo and Adelaide Bank
7. Macquarie Bank
8. Bank Australia
9. Suncorp Bank
10. ING Australia

Disagree, they could make it safe
On Thu, 19 Dec 2024 at 11:49 am, erspark2002 @.***>
wrote:

In hindsight, it actually makes sense for bank to not provide a public
API, as poorly implemented commercial software using these API's would be a
major security risk.
Xero accounting software is ACCC accredited, and supports a wide range of
Australian banks for automated bank feeds.

ANZ (Australia and New Zealand Banking Group)
2. Commonwealth Bank of Australia (CommBank)
National Australia Bank (NAB)
Westpac Banking Corporation
Bank of Queensland (BOQ)
Bendigo and Adelaide Bank
Macquarie Bank
Bank Australia
Suncorp Bank
ING Australia

—
Reply to this email directly, view it on GitHub
https://github.com/twang2218/node-cba-netbank/issues/17#issuecomment-2552545581,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE2X4ZZMY2OYUJX72HXZUD2GIJXZAVCNFSM6AAAAABT33A27OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJSGU2DKNJYGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

Yeah I also disagree.
They'll be required to do so anyway under the Consumer Data Right:
https://www.accc.gov.au/by-industry/banking-and-finance/the-consumer-data-right
https://consumerdatastandardsaustralia.github.io/standards/


undefined method `refresh_line'

hi animotto!
Ho to fix an error when i try to run any script.
✿ Error: simlink [4]

4. C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:79:in `block (2 levels) in <top (required)>'
3. C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:40:in `script_run'
2. C:/ta-hackers/ta-hackers-2.0/lib/cli/sandbox.rb:45:in `log'
1. C:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/sandbox-ruby-0.1/lib/sandbox/shell.rb:109:in `puts'

=> undefined method `refresh_line' for Reline:Module
#<Thread:0x00000220666ec7a0 C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:79 run> terminated with exception (report_on_exception is true):
/script ▸ C:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/sandbox-ruby-0.1/lib/sandbox/shell.rb:109:in `puts': undefined method `refresh_line' for Reline:Module (NoMethodError)

      Readline.refresh_line if @reading
              ^^^^^^^^^^^^^
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/sandbox.rb:49:in `error'
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:60:in `script_log_backtrace'
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:45:in `rescue in script_run'
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:34:in `script_run'
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:79:in `block (2 levels) in <top (required)>'
C:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/sandbox-ruby-0.1/lib/sandbox/shell.rb:109:in `puts': undefined method `refresh_line' for Reline:Module (NoMethodError)

      Readline.refresh_line if @reading
              ^^^^^^^^^^^^^
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/sandbox.rb:45:in `log'
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:40:in `script_run'
        from C:/ta-hackers/ta-hackers-2.0/lib/cli/context/script.rb:79:in `block (2 levels) in <top (required)>'```

To fixed this need to comment or remove 109 line in C:\Ruby\lib\ruby\gems\3.2.0\gems\sandbox-ruby-0.1\lib\sandbox\shell.rb


[VS Code] Auto-commenting line

I'd like to request feature where someone would create a line and code in it and at the end of that line would be added //By: Username at Day,hour
And if someone would edit it after that would look like this:
//By: vs_live_share_getUsername() Edited by: Username at Day,hour
And so on, if someone edits afterwards then add him in the end of the comment.
Is it possible ?
Comment type // for c++ or # for python and etc would be based on file extension

@horsetaur Thanks for the feedback! We’ve been tracking #680, which is meant to help provide a log of all activity in a session (e.g. edits, guests joined, server shared, etc.). But, we can use this issue to track the inline “blame” experience, specifically for code changes.
This is definitely something we’ve heard a lot of feedback on, and want to address in the near future. We’ll keep you posted on any progress, so stay tuned. 👍


Add Discussion Area to Github

Hey,
i would love if you could add Discussion Area to your Github Site, that we can ask Questions there, instead of needing to open an Issue. It would be nice and make it easier for us too.

Ähhm #639


Add thanks Sanket

This is just a trivial patch to recognize that the code here was originally hacked together by Sanket, and so we can get an ack on github to show he is okay with me stealing the code and ok with me leaving his name in the author list.
Thanks Sanket!
Fix: #1

CC @sanket1729

Thanks for acknowledging my work :)


Features added:

Hello,
Great project, I was trying to do something like that until I find yours. In order to solve all my needs I had to add a few things. Let me know what you think.
Unfortunately I couldn't make the test cases for that because I don't have experience with pytest, but I'm adding examples of how it works.
Things that I have added:

Custom stop words option in the remove_stop_words method
Added the method "remove_numbers" to remove the numbers
Added the method "replace_custom_regex" to remove a custom regex in the text.

Sample code to test my changes:
from cucco.cucco import Cucco

from cucco.config import Config
cucco_config = Config(language='en')
c = Cucco(config=cucco_config)

# Replace numbers but those that are only numbers, not numbers between letters.
c.replace_numbers("this 3333 i3s a text with the number 2")

# Removing custom regex, for example all #foo and @bar
import re
regex = re.compile(r"[#@]\w+", re.IGNORECASE)
c.replace_custom_regex(regex=regex, text= "Test a string #foo to replace @bar")

# Removing custom stop words

# This is the default one, with all stop words:
c.remove_stop_words("Test to remove stop words")

# This is with a custom set of stop words (in case that you want to use your own set):
c.remove_stop_words("Test to remove stop words", custom_stop_words=['test', 'to'])

Sample code with the console output:
In [1]: from cucco.cucco import Cucco
   ...:
   ...: from cucco.config import Config
   ...:
   ...: cucco_config = Config(language='en')
   ...:
   ...: c = Cucco(config=cucco_config)
   ...:

In [3]: # Replace numbers but those that are only numbers, not numbers between letters.
   ...: c.replace_numbers("this 3333 i3s a text with the number 2")
Out[3]: 'this i3s a text with the number'

In [4]: # Removing custom regex, for example all #foo and @bar
   ...: import re
   ...: regex = re.compile(r"[#@]\w+", re.IGNORECASE)
   ...: c.replace_custom_regex(regex=regex, text= "Test a string #foo to replace @bar")
   ...:
Out[4]: 'Test a string  to replace '

In [5]: # This is the default one, with all stop words:
   ...: c.remove_stop_words("Test to remove stop words")
   ...: 'test remove stop words'
   ...:
Out[5]: 'test remove stop words'

In [6]: # This is with a custom set of stop words (in case that you want to use your own set):
   ...: c.remove_stop_words("Test to remove stop words", custom_stop_words=['test', 'to'])
   ...: 'remove stop words'
   ...:
Out[6]: 'remove stop words'

Thank you for your contribution. I really appreciate.
It will take me some time to review it. As you can see, this is still a single guy project and these days I'm a bit busy. But for sure I will review it and try to add your great suggestions.
Cheers.


Add patient ID

In our anonymization method, the DICOM patient ID tag is removed. We wrote some code to set patient ID to the patient variable defined in the "enter user settings" code.
set patient ID to patient variable (defined above) for all dicom files in src
for series_dir in [series_dir for series_dir in os.listdir(src) if os.path.isdir(os.path.join(src, series_dir))]:
for dcm_file in [dcm_file for dcm_file in os.listdir(os.path.join(src, series_dir)) if dcm_file.endswith(".dcm")]:
ds = pydicom.dcmread(os.path.join(src, series_dir, dcm_file))
ds.PatientID = patient
ds.save_as(os.path.join(src, series_dir, dcm_file))

Thank you for this. It is a good idea to ensure that all of the required DICOM header information is present prior to attempting to use the existing pipeline. I'll make a list of required DICOM header information and write a quick code block to check that it is all defined. If not, we can use code such as yours to add the required information.


add chat cleanup and future discussion

For future discussion, would @bast want to host this and does the date fit?

Thanks! It fits well.


feat  : add @stdlib/assert/is-nonnegative-finite

Resolves #1344 .
Description
This PR adds the package @stdlib/assert/is-nonnegative-finite.
The package implements assertion for nonnegative number except positive infinity.
Alias: isNonNegativeFinite
Related Issues

resolves [RFC]: add @stdlib/assert/is-nonnegative-finite #1344

Questions
I have used the @stdlib/asset/is-finite in order to achieve the desired result. Is this considered good practice, what are the guidelines for using internal packages, when and where is it ideal.
Checklist

[x] Read, understood, and followed the contributing guidelines.

@stdlib-js/reviewers

@Planeshifter Hi, apologies, I forgot to correct the licensing dates, I have signed and commited your suggestions.
Also, thanks for letting me know about the internal dependency convention,

Hi, I signed off and commited your messages. Thanks for highlighting what I should keep in mind from next time on.
I had a question though...

// eslint-disable-line stdlib/no-redeclare

I did not understand what this comment is for and where should I use this. googling about this only results in "Disabling an eslint feature" but why and when would we want to do so?

I just noticed your edit regarding merging commits from the file section.
I will keep that in mind from next time on

I just noticed your edit regarding merging commits from the file section.
I will keep that in mind from next time on

@marsian83 Will merge once CI passes.


Make epaint pixel_per_point warning only trigger on same float diff as fonts is doing

Apparently the font implementation uses a distance check to decide if the font(or whatever) need recalculations, after dpi changed:
https://github.com/emilk/egui/blob/8d4de866d4da1835b31e85f9068a5257e6ccbccb/crates/epaint/src/text/fonts.rs#L381-L382
This leads to warnings when the pixel_per_point diff is very low and spams the log. (<- this happens for me if i resize my window on kwin, e.g. maximize it)
(I don't want to debate if the float difference generally makes sense, so if you want to rework that instead just close this pr)

This seems to be fixing a symptom rather than the problem. Have you investigated why the float difference happen at all? I don't hit this assert.

I don't know if i want to dig into the details of winit & x11, but this value is reported by winit, and it's definitely changing by this small amount.
But besides that, if for whatever reason the DPI changes by a very small amount, what does egui do to handle it?
my code snipset shows that it ignores small differences. the warning that I i fixed in this pr does not. So these two code segments behave different on the same logic, or not?

But where does the change come from? Are you storing a Galley between frames (you shouldn't)?
If you create the galley in one frame, and then tesslleate, then I can't see how the pixels_per_point could change.
Have you tried latest master?

But where does the change come from? Are you storing a Galley between frames (you shouldn't)?

Well the Galley comes from from the font:
https://github.com/emilk/egui/blob/327f5994079181f8ef2e8918722e6eebcbc676e2/crates/epaint/src/text/fonts.rs#L563-L567
I honestly dunno how exactly to debug if these Galley are stored between frames, but a GalleyCache sounds like the implementation does that?
I have to say I don't have the insight into egui, the docs for the Galley also only say:
"Needs to be recreated if the underlying font atlas texture changes, which happens under the following conditions: [...]"
(https://docs.rs/egui/latest/egui/struct.Galley.html)
I am not manually recreating my fonts if pixels_per_point change, but the doc sounds like it's doing that internally anyway in crate::text::Fonts::begin_frame
Then bcs of:
https://github.com/emilk/egui/blob/8d4de866d4da1835b31e85f9068a5257e6ccbccb/crates/epaint/src/text/fonts.rs#L381-L382
The pixels_per_point change is ignored (because of the floating point difference)
=> galley.pixels_per_point
but the line in this pr compares it again self.pixels_per_point which IS changed
(galley.pixels_per_point: 1.16666663 vs. self.pixels_per_point: 1.16666675 (so very very small => galley is not updated in https://github.com/emilk/egui/blob/8d4de866d4da1835b31e85f9068a5257e6ccbccb/crates/epaint/src/text/fonts.rs#L381-L382
but the warning is still displayed
)
I'm indeed not using latest master, if you say Galley is never cached between frames, then it's indeed a bug in my application. I am using custom fonts, but i am not sure which font is triggering this issue.

Does
-        let pixels_per_point_changed =
-            (fonts_and_cache.fonts.pixels_per_point - pixels_per_point).abs() > 1e-3;
+        let pixels_per_point_changed = fonts_and_cache.fonts.pixels_per_point != pixels_per_point;

solve your problem?
I really don't see the point of the 1e-3 epsilon there

Yes that works just as good :+1:


Fix Decktape / video building

Fixes https://github.com/galaxyproject/training-material/issues/3234

Maybe squash this merge.

It's working here https://github.com/galaxyproject/training-material/actions/runs/3330593040/jobs/5509278409

omg finally fixed! What a weird issue, Thanks @hexylena


Update USERS.md

add Cilium user "PostFinance AG"
Please ensure your pull request adheres to the following guidelines:

[x] For first time contributors, read Submitting a pull request
[ ] All code is covered by unit and/or runtime tests where feasible.
[ ] All commits contain a well written commit description including a title,
description and a Fixes: #XXX line if the commit addresses a particular
GitHub issue.
[ ] All commits are signed off. See the section Developer’s Certificate of Origin
[ ] Provide a title or release-note blurb suitable for the release notes.
[ ] Thanks for contributing!

Fixes: #issue-number
Add "PostFinance AG" to USERS.md

Hi @imathu, thanks for the PR! Could you Sign-Off the commit please?

Hi @imathu, thanks for the PR! Could you Sign-Off the commit please?

Re-opened as #14831.