HTTP Error 403: FORBIDDEN (caused by HTTPError()) downloading certain UDEMY video

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.04.14. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2017.04.14

Before submitting an issue make sure you have:

[X] At least skimmed through README and most notably FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[X] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

PS E:\UDEMY Courses\ITIL Foundation COMPLETE ITIL Foundation Exam Training> .\youtube-dl.exe -v -u<EMAIL_ADDRESS>-p ********* https://www.udemy.com/draft/470634/learn/v4/overvie
w
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-u', 'PRIVATE', '-p', 'PRIVATE', 'https://www.udemy.com/draft/470634/learn/v4/overview']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.04.14
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] Logging in as<EMAIL_ADDRESS>[udemy:course] draft: Downloading webpage
[udemy:course] 470634: Downloading course curriculum
[download] Downloading playlist: 470634
[udemy:course] playlist 470634: Collected 121 video ids (downloading 121 of them)
[download] Downloading video 1 of 121
[udemy] Downloading login popup
[udemy] 2771478: Downloading webpage
[udemy] 2771478: Downloading lecture JSON
[udemy] 2771478: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: FORBIDDEN (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest versio
n; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpykl6x8ju\build\youtube_dl\extractor\common.py", line 498, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpykl6x8ju\build\youtube_dl\YoutubeDL.py", line 2100, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

PS E:\UDEMY Courses\ITIL Foundation COMPLETE ITIL Foundation Exam Training>
...
<end of log>

Multiple videos from single Udemy course: https://www.udemy.com/draft/470634/learn/v4/overview

Trying to download my paid Udemy content, all videos have downloaded correctly except the following URL:https://www.udemy.com/draft/470634/learn/v4/overview  I have tried comming back on a different day to see if the course was locked or updated etc.  No success.  Attached dump if needed, again all other videos ~50 downloaded with no incident using same creds, powershell as admin and same parent directory.
Udemy_HTTP_Error_403_dump.txt

For any further investigations you have to provide account credentials.

Same here, but having:
"ERROR: Unable to download JSON metadata: HTTP Error 406: NOT ACCEPTABLE (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output."

So after looking further into my own issue, I found that the Udemy error was caused because there was a "draft" (the course was being updated) I have found that by deleting the "draft" portion of the URL, the download worked.  I.e:
https://www.udemy.com/draft/470634/learn/v4/overview (does not work)
https://www.udemy.com/470634/learn/v4/overview (does allow download without errors of all content)
this may be useful for other users who are having issues downloading wile the course is in a "draft" state.
My issue has been resolved at this time.


If all the fields in a closed record typedesc are readonly the record type itself should be a subtype of readonly

Description:
$title.
The spec says

If a record type descriptor is closed and every individual-type-descriptor specifies readonly, then it describes a type that is a subtype of readonly: a shape belongs to the type only if its read-only bit is set.

Steps to reproduce:
type Person record {|
    string name;
    int id;
|};

type Student record {|
    readonly string name;
    readonly int id;
|};

public function main() {
    Student st = {
        name: "Maryam",
        id: 1234
    };

    Person & readonly pr = st; // Should work

    readonly rd = st; // Should work
}

Same for objects too.

Depends on #23420 for objects.


Embed CT log public key for SCT verification

This way, we don't have to depend on users running init and we can error out if SCT verification fails.
Ref https://github.com/sigstore/cosign/issues/604
Signed-off-by: Priya Wadhwa<EMAIL_ADDRESS>

Nice. Longer term we'll probably want a way to embed the entire latest TUF root when we start to use real delegations for this.


Testing, please ignore. (ceggxf)

This is a bug bounty test.. Please DO NOT APPROVE THIS! (ceggxf)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.ceggxf jaefc seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Mapped existing sts caller specified role name

Fixes #120
When using a federated user session in conjunction with
the -r role assumption option, some of the fidelity of that
initial session is lost (ex: caller-specified-name from
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html#principaltable)
This will conditionally check, if a new role is asked to be assumed,
if there is already an existing role specified and if so, it will
pass it through to the new session to allow for easier auditing
of users as they traverse more complex IAM role delegation structures.

I like the idea of providing this for convenience.  Should this be done by default or should it be an option?  Should we also provide a way to override the session name?
You call out being able to use it for auditing but I would be careful with that.  Ultimately the session name is still controlled by the client so unless where this is running is considered trusted they can always set their own session name with a custom client.

I would say default is acceptable since the alternative is just a random
string.
Re the trust issue, understood and thanks. The initial session for all of
our roles only come from ADFS which controls that. There are some
additional attributes of the audit log that allow proving that they match
up.
On Sun, Jul 29, 2018 at 10:55 AM Matthew Landis<EMAIL_ADDRESS>wrote:

I like the idea of providing this for convenience. Should this be done by
default or should it be an option? Should we also provide a way to override
the session name?
You call out being able to use it for auditing but I would be careful with
that. Ultimately the session name is still controlled by the client so
unless where this is running is considered trusted they can always set
their own session name with a custom client.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-sigs/aws-iam-authenticator/pull/121#issuecomment-408687029,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAWanP613lCdEa-Xfq3RImzHkc8O6e3Sks5uLdsHgaJpZM4VN59C
.

Hey thanks for this PR. We're not going to merge this PR because the session can be an arbitrary string, this makes that audit not trustworthy. While it could be helpful for debugging, its not something we want to suggest should be trusted for an audit.
/close

/close

The session string can be arbitrary but the other component can not.  With a completely opaque token as it is now, you ALWAYS have to go back to something like CloudTrail to try and associate a user with a given attribute.  I personally don't think that carrying forward the same session name attribute that you get if you aren't federated is that much less secure than the alternative.
Would it be acceptable to map it to a configuration flag that a user has to opt into?

I think an off-by-default option would be a good solution. The flag could be named something like --forward-session-name and stating in the help text that this forwards the caller-specified session name. Also a README note also specifying that this should not be used for access control. What we do not want to happen is for someone to rely on this for auditing or ACLs based on the session name.
/reopen

Sounds fair, thanks. I'll make some adjustments
On Thu, Aug 16, 2018 at 3:01 PM k8s-ci-robot<EMAIL_ADDRESS>wrote:

@micahhausler https://github.com/micahhausler: you can't re-open an
issue/PR unless you authored it or you are assigned to it.
In response to this
https://github.com/kubernetes-sigs/aws-iam-authenticator/pull/121#issuecomment-413667307
:
I think an off-by-default option would be a good solution. The flag could
be named something like --forward-session-name and stating in the help
text that this forwards the caller-specified session name. Also a README
note also specifying that this should not be used for access control. What
we do not want to happen is for someone to rely on this for auditing or
ACLs based on the session name.
/reopen
Instructions for interacting with me using PR comments are available here
https://git.k8s.io/community/contributors/guide/pull-requests.md. If
you have questions or suggestions related to my behavior, please file an
issue against the kubernetes/test-infra
https://github.com/kubernetes/test-infra/issues/new?title=Prow issue:
repository.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-sigs/aws-iam-authenticator/pull/121#issuecomment-413667308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAWanCIq3JG-ewhdlwEjPGqQRhQGWfJZks5uRc-YgaJpZM4VN59C
.


Ktrain based text classifier (internet off) - Two problems

Hi,
Env details :
Linux : CENTOS 7.5
Python 3.7
Installed tensorflow 2.1
ktrain 0.14.5
transformers 2.8
Problem 1
We had built the model in DEV server and deployed the model in PROD for prediction. Noticed the following issues.
The prediction script looks at cache directory for the OS USER who created the model in DEV env. Even if the DEV OS user doesn't exist in PROD....it is expecting the same user. I had to create the same OS user in PROD and then copied the CACHE contents from DEV to PROD  and grant access to contents to all others. Then prediction worked. Why this behavior ?

Problem 2
 Tried building model afresh in PROD (internet off). Got the following error.

OSError: Can't load weights for 'albert-base-v2'. Make sure that:

'albert-base-v2' is a correct model identifier listed on 'https://huggingface.co/models'

I can download the prebuilt model as said in FAQ.
Can you please let me know how can i reference the three files (h5,json and vocab) in the training phase.
Here is the code I have
t = text.Transformer(paramDict['modelName'], maxlen=25, class_names=dataList[4])
trn = t.preprocess_train(dataList[0], dataList[2])
val = t.preprocess_test(dataList[1], dataList[3])
model = t.get_classifier()
learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=6)
learner.fit_onecycle(8e-5, numEpochs)
Thanks
Sridhar

Hello,
Note that there is a long-standing bug in the transformers library that causes problems with loading files from cache when there is no internet connectivity.  It seems to affect TensorFlow rather than PyTorch.  This is why you're seeing the ERROR: OSError: Can't load weights for 'albert-base-v2'..  Hopefully, they'll fix it soon.  In the meantime,  the steps in the FAQ describe a workaround solution that avoids these errors.
Please find details below:
Problem 1:
The path to the cache directory is constructed by the transformers library, not ktrain.  It is supposed to look at the cache folder of the current OS user, not the user on the development server.  I believe it uses this to do so:
from torch.hub import _get_torch_home
print(_get_torch_home())

Not sure why the above would print the home directory of USER on dev machine, if that's what's happening.
I have never experienced the problem you're describing.   I will say that people train models on dev servers and Google Colab and apply these models on different machines all the time with no problem.  I did notice that you're using an older version of transformers and ktrain.   Perhaps, this is a bug in the version of transformers that you're using.  Have you tried upgrading transformers (and also ktrain) to the latest version to see if the problem still occurs?  If not, please do so and let me know if the problem persists.  The current version of transformers is v3.0.2 and the current version of ktrain is 0.18.3.
I believe the path is constructed by the transformers library using:
Problem 2:
I have updated the FAQ with more details what to do.  Basically, you download the three files to a folder (e.g., my_model_files), copy the folder to the machine with no public internet access, and replace MODEL_NAME with the path the folder:
t = text.Transformer('my_model_files', maxlen=25, class_names=dataList[4])
trn = t.preprocess_train(dataList[0], dataList[2])
val = t.preprocess_test(dataList[1], dataList[3])
model = t.get_classifier()
learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=6)
learner.fit_onecycle(8e-5, numEpochs)

Also, when deploying a Hugging Face model using only a Predictor instance:
p = ktrain.load_predictor('path_to_predictor_folder')
p.predict(data)

... everything should work fine as long as the cache folder is populated correctly (i.e., vocab file is in the cache).  I just tested this on a machine with no internet connectivity and everything worked perfectly.


Where docker-machine copies the cert keys to remote vm with azure driver ?

Couldn't find any doc or related issues about this.

Depends on the OS.  Frequently /etc/docker


Smaller/more performant?

I was thinking maybe it could be possible to use Set instead to do the same thing...
A benchmark for it maybe?
var a = ['a', 'b', 'c', 'd'];
var b = ['b', 'c'];

var set = new Set(a)
b.forEach(a => set.delete(a))
Array.from(set) // => ['a', 'd']

Also in many situations it's wasteful to cast the set into an array... ppl only do it to keep the same signature and never used set in the first place
curious to see a benchmark for this native solution

Also in many situations it's wasteful to cast the set into an array... ppl only do it to keep the same signature and never used set in the first place

@jimmywarting so do you want to add another function which you call like this:
anotherDiffFn(setToRemoveValuesFrom, arrayOfThingsToRemove)

?

I was thinking more in terms of:
// index.js
export default function diff (a, b) {
  const set = new Set(a)
  b.forEach(a => set.delete(a))
  return set
}

import diff from 'arr-diff'

var a = ['a', 'b', 'c', 'd']
var b = ['b', 'c']

console.log(diff(a, b))
//=> Set(2) {'a', 'b'}

doe it would be breaking change...


The new code coverage formats also for force:apex:test:*

I think the new --coverageformatters option is a valuable addition to the SFDX cli. But it would be really nice to also add it to force:apex:test:*
For example:

sfdx force:apex:test:run --coverageformatters cobertura

This would be valuable to scratch org build deployments in pipelines. So you can forward the code coverage to your CI/CD system of choice.

I have a Salesforce CLI plugin that can convert the "json" coverage format available in the Salesforce CLI deploy and test run commands into a valid "cobertura" format.
See https://github.com/mcarvin8/apex-code-coverage-transformer


Improve error messaging for bad reqs

Link to reproduction
https://github.com/payloadcms/plugin-sentry
To Reproduce
Send forgot password request without body:
const req = await fetch('http://localhost:3000/api/users/forgot-password', {
  method: 'POST',
  credentials: 'include',
  headers: {
    'Content-Type': 'application/json',
  },
})

Send unlock req without body:
const req = await fetch('http://localhost:3000/api/users/unlock', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
})

This is setup and can be reproduced in the sentry plugin demo.
Describe the Bug
Both of these bad requests return a TypeError:

TypeError: Cannot read properties of undefined (reading 'toLowerCase')

We should update to throw a more informative error.
Payload Version
1.9.2

@JessChowdhury The link to reproduction is broken

@Victory-M the repo was private but has now been made public.

@jmikrut kindly assign me to this


testcafe-browser-provider-saucelabs marked as dependency?

The package testcafe-browser-provider-saucelabs is marked as a dependency when installing the package through NPM, which makes the slider install a whole lot of code (like Babel, a browser and similar stuff) I'm pretty sure I won't need. The dependency graph for my project literally (figuratively) exploded.
Shouldn't it be marked in devDependencies instead, or am I missing something?

Thanks, it's updated


Support for array of macaddr

What happens?
Hello,
I have a postgresql table with an array column of mac addresses. I can't read the table from duckdb. I have the following error: Error: INTERNAL Error: Unsupported type for ColumnDataCollection::GetCopyFunction
It seems that macaddr is not supported here. It would fine to have a conversion to varchar.
To Reproduce
In postgresql:
CREATE TABLE test
(
    mac_addresses macaddr[]
);

TRUNCATE test;
INSERT INTO test (mac_addresses)
VALUES ( '{00:00:00:00:00:00, 00:00:00:00:00:01, 00:00:00:00:00:02}'::macaddr[] );

In duckdb:
INSTALL
postgres_scanner;
    LOAD
postgres_scanner;

CALL postgres_attach('***);

SELECT * FROM test;
-- gives Error: INTERNAL Error: Unsupported type for ColumnDataCollection::GetCopyFunction

### OS:

macOS

### PostgreSQL Version:

15

### DuckDB Version:

0.7.1

### DuckDB Client:

CLI

### Full Name:

Kamil Caglar

### Affiliation:

Serenicity

### Have you tried this on the latest `master` branch?

- [X] I agree

### Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

- [X] I agree

would this work ? it be VARCHAR ... this is my POC code output ...

Thanks for the report! This should be fixed now in https://github.com/duckdb/postgres_scanner/pull/111.


feat: create ImagePlaceholder component

This is a stacked PR, please review the linked diff

PR Stack

PR
Title
Merges Into
diff

1
refactor: #40
design-updates
diff

2
ImagePlaceholder
#40
diff

This PR removes the placeholder element fro the Gallery component and uses the new ImagePlaceholder component instead. It also removes the placeholder text from the Dialogue component and its corresponding styles, since those are now housed in the ImagePlaceholder.
 But I figured we would take care of all spacing/styling once the UI has been fully added in. That way we don't end up overwriting each other until we know where everything (relatively) goes.


fixing dependency error in build

What type of PR is this?
(bug/feature/cleanup/documentation/test/refactor)
Fix
What this PR does / why we need it?
adds missing import for metav1
00:03:08 30.42 osde2e/rbac_permissions_operator_tests.go:83:50: undefined: metav1
Which Jira/Github issue(s) this PR fixes?
Fixes #
Special notes for your reviewer:
Pre-checks (if applicable):

[ ] Tested latest changes against a cluster
[ ] Included documentation changes with PR

/approve


Multi await feature for intentional blocking

There was a request for blocking feature for Multi and Single
Multi.interval(20, TimeUnit.MILLISECONDS, Executors.newSingleThreadScheduledExecutor())
        .limit(5)
        .forEach(sum::addAndGet)
        .await();

Multi.interval(20, TimeUnit.MILLISECONDS, Executors.newSingleThreadScheduledExecutor())
        .limit(5)
        .reduce(Long::sum)
        .await();

I guess this needs some broader discussion, it kind of feels like introducing an antipattern. But the truth is other impls has similar feature and users will be expecting it.
Signed-off-by: Daniel Kec<EMAIL_ADDRESS>

Can you rebase against master ? you're missing one of the checks from the new infra.

@tomas-langer had proposed Single implementing CompletionStage which is seems to be quite nicer, so Multi.forEach can returnSingle<Void>, await methods are part of single on one place only and its possible to return Single on the apis where completables are expected

Can I ask @romain-grecourt and @tomas-langer for review? @Verdent is blocked by this with #1646 Question is if Single implementing CompletionStage is preferable or not, Imho I like it as it makes all apis returning Single more comfortable.

@danielkec Yeah, Let's go ahead with Single implementing CompletionStage, we can leverage it in our APIs where we return a CompletionStage.

Awaits are now preserved when chaining CompletionStage methods:
        testMulti()
                .forEach(sum::addAndGet)
                .whenComplete((aVoid, throwable) -> {})
                .thenRun(() -> {})
                .thenAccept(aVoid -> {})
                .await();

This seems like a massive breaking change just to get a wrapper for a timed CompletableFuture.get.
onCancel implementation is incompatible in its behaviour with superclass. Multi.forEach can miss cancellation indefinitely. Dangling Subscribers are possible.

@olotenko pls elaborate, is it order of?:
                    subscription.request(Long.MAX_VALUE);
                    single.onCancel(subscription::cancel);

Yes, but don't you see the whole design of the change is rickety?
A plain static method for two flavours of await would serve the purpose. I really don't see the value of rewriting CompletableFuture and changing how all the reactive pipelines everywhere behave.


Occurring EResult 93 constantly (every 5 minutes)

The time in saliencheat is 3 hours ahead
How can I fix this?

Now it's spaming all these problems

You're getting these errors because of weekly Steam maintenance. The time being 3 hours ahead doesn't matter, it's just because Steam is in a different timezone. The errors will go away eventually.


Event for loading ViewModel data on startup?

What event should I use in my viewmodel to load persistent data when my mainpage initially loads? Both the PageLoaded and OnNavigatedToAsync events will continue to reload the data if the page is navigated away and then back again, and the OnLauched and OnActivated events cannot be used. I have a json file containing some data which the user inputted, and is saved when the user closes the app, but need to be loaded just once when the app is started again.

I have been using a static object that saves and loads from the SettingsService... I then created a baseclass for my viewmodels which also exposes the ViewModelBase from T10, and created a Property which holds a reference to that SettingService property.  Works quite well..  I am sure there are better ways but for the moment that is how I am doing it... SessionState property might be another solution to this as well..  It is located in the ViewModelBase within T10

You should abstract the logic surrounding your data to a dedicated class, something like Services/DataService.cs and in your ViewModel.OnNavigatedTo() call this.Data = service.GetData() every single time, no matter what. Then implement caching inside the DataService.GetData() method, not inside the view-model. Then, you can call and get the data over and over without special logic, and you can call it from other view-models without worrying about caching inside a one (or more) view-model. You were already doing it right with ViewModel.OnNavigatedTo(), but the best pproach is with a repository service like DataService to handle the hard work. The caching, @mvermef is correct, is most easily handled in a static variable/field/property. Doing it this will also allow you to properly handle Resume when you cannot rely on the cache being there, but the DataService can check for that and simply repopulate it. When you navigate to pages in your app, navigate then with only the key that you will feed to the DataService, this also simplifies your workflow and increases your success rate to handle suspending/resume. It's all so easy. I should make some training videos, I see that now.

Thank you for your help. I've used a Setting Service with a singleton before, but this time I was trying to keep it a little simpler and I don't anticipate needing to reuse the data in other viewmodels, so I just used a bool and the OnNavigatedToAsync method to call my Deserializer method. You can find the complete inelegant solution here: http://stackoverflow.com/questions/36994792/how-to-save-and-load-an-observablecollection-on-app-suspend-and-start/37096217#37096217
Next time I'll try abstracting the data out using a Dataservice class like you suggest.

Inelegant solutions are still solutions. :)


Add codium bot

@CodiumAI-Agent /describe

@CodiumAI-Agent /improve

PR Code Suggestions ✨

CategorySuggestion                                                                                ScoreGeneral

Restrict the workflow trigger to specific branches to reduce unintended executions

Use a more specific event type for triggering the workflow to avoid unnecessary executions, such as limiting it to specific branches or conditions.
.github/workflows/auto-comment.yml [3-5]
 on:
   pull_request:
     types: [opened]
+    branches: [main]

Suggestion importance[1-10]: 8
Why: Adding branch restrictions to the workflow trigger is a practical improvement that reduces unnecessary executions and enhances efficiency. The suggestion is accurate and directly applicable to the code.
8
Possible issue

Verify the presence and correctness of the required GitHub secret to prevent workflow failures

Ensure that the secrets.SEMOSS_GITHUB_TOKEN secret is correctly configured in the repository settings to avoid runtime errors during workflow execution.
.github/workflows/auto-comment.yml [18]
+token: ${{ secrets.SEMOSS_GITHUB_TOKEN }}

-

Suggestion importance[1-10]: 7
Why: Ensuring the correct configuration of the GitHub secret is crucial for the workflow to execute successfully. However, this suggestion is not actionable in code and requires manual verification, which slightly reduces its impact.
7

Changelog updates: 🔄
2024-12-13
Added

Automated workflows to add comments on PRs for description, review, and improvement using CodiumAI bot.
Workflow to comment on PR closure with a changelog update request if merged.

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'


WebGPU compilation errors: "function parameter of pointer type cannot be in 'storage' address space"

What happened?
Several of our tests are failing to compile using the WebGPU compiler target, during compilation from SPIR-V to WGSL using Tint.
From tests/e2e/xla_ops/:

clamp.mlir
compare.mlir
convert.mlir
finite.mlir
maximum.mlir
minimum.mlir

From tests/e2e/tosa_ops/:

equal.mlir
greater.mlir
greater_equal.mlir
max_pool.mlir
table.mlir

The error message from Tint is:
Tint reported 1 error(s) for a SPIR-V program, see diagnostics:
error: function parameter of pointer type cannot be in 'storage' address space

Steps to reproduce your issue
Either add a test suite to tests/e2e/xla_ops/ or tests/e2e/tosa_ops/ with
  TARGET_BACKEND
    "webgpu"
  # Only test compilation for now, the WebGPU driver is not stable/tested yet.
  # DRIVER
  #   "webgpu"

OR just run the compiler on one of those files directly, e.g.
iree-compile --iree-hal-target-backends=webgpu \
  --iree-input-type=mhlo tests/e2e/xla_ops/clamp.mlir \
  --iree-hal-dump-executable-sources-to=/tmp/iree/webgpu_clamp/ \
  --iree-hal-dump-executable-intermediates-to=/tmp/iree/webgpu_clamp/ \
  --iree-hal-dump-executable-binaries-to=/tmp/iree/webgpu_clamp/ \
  -o /dev/null \

Version information
Tip of tree, https://github.com/iree-org/iree/commit/29b1e8832fd87080716545b0337276e0b3aed636
Additional context
At least some of these tests successfully compile when they are modified to only use i32 and f32 types, and not i8 or i16.
This gist has the .spvasm that IREE feeds to Tint for i8 (fails) and for i32 (succeeds): https://gist.github.com/ScottTodd/c01f1591d8ff2ed38896adcd609ddd65

From @dneto0: https://dawn-review.googlesource.com/c/dawn/+/109820 should fix this.

I'm still seeing this issue after updating Tint. I've only seen it affect unit tests (and not full programs) so far though.


Add topic "web shell tools"

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the master branch of freeCodeCamp.
[ ] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

We are marking this PR invalid for Hactoberfest as it does not follow the recommended guidelines below:

Quality Standards
In line with Hacktoberfest value #2 (Quantity is fun, Quality is key), we have provided examples of the quality standards we encourage. This applies mainly to beginners.

PRs that are automated e.g. scripted opening PRs to remove white space / optimize images.
PRs that are disruptive e.g. taking someone else's branch/commits and making a PR.
PRs that are regarded by a project maintainer as a hindrance vs. helping.
Something that's clearly an attempt to simply +1 your PR count for October.

Learn more at https://hacktoberfest.digitalocean.com/details
Parts of this is taken from https://www.packtpub.com/mapt/book/networking_and_servers/9781849519489/11/ch11lvl1sec74/creating-web-backdoors


Time series example is broken

If I run timeseries_weather_forecasting.py with the latest version of Keras, I get the following error:
AttributeError: module 'tensorflow.keras.preprocessing' has no attribute 'timeseries_dataset_from_array'

Just now tested. It works in tf==2.5.0 and tf==2.6.0-rc1.
Did not test installing a keras version and whatever TF it includes.

Hi @markrogersjr ,
The example tutorial working fine now. Please refer to attached gist.
Please cross check and close the issue if found resolved else let us know if any issue still there.
Thanks!


Not able to run repo locally

Am not able to run the repo locally , can someone tell how to do this  ?
Getting this  :

Am not able to run the repo locally , can someone tell how to do this ?

facing the same issue

same issue here

same issue here

step1.   Comment the code line //@import "react-image-crop" ,in plugins.scss
step2.  npm install isomorphic-fetch --include-dev
step2. npm start

step1. Comment the code line //@import "react-image-crop" ,in plugins.scss step2. npm install isomorphic-fetch --include-dev step2. npm start

This works fine for me. Thanks @hellereye


Epic - SQL Managed Instance

mssql_mi

[x] sqlmi module based on arm deployment (create, update)
[x] example (subnet prereq: service delegation, NSG, Routes)
[x] destroy part (sqlmi, Virtual Cluster)
[x] Administrators
[x] Failover Groups
[x] Encryption Protector & Keys #204
[ ] Restorable Dropped Databases
[ ] Security Alert Policies
[ ] Vulnerability Assessments
[ ] Instance Pool
[ ] Server Trust Groups
[ ] Restore MI from another MI backup

managed db

[x] managed database (brand new db: Default)
[x] managed database (PointInTimeRestore)
[ ] managed database (RestoreExternalBackup)
[ ] managed database (Recovery)
[x] managed database (RestoreLongTermRetentionBackup) #184
[x] Backup Short Term Retention Policies
[x] Backup Long Term Retention Policies #184
[ ] Security Alert Policies
[ ] Vulnerability Assessments

Closing in favor or the native TF provider.


Want to add drawable icons insteadof colorful dots

I want to add some drawable icons instead of colorful dots to represent an event.
Is this possible ?
I tries this, but it didn't worked
new Event(R.drawable.ic_checked, timeInMillis, "Event at " + new Date(timeInMillis)),

This is a duplicate of #181. Have a read through and let me know if you don't understand. Short version is, not currently supported. If you want it, you can fork the code and the part where the dot is drawn, you can just draw your icon instead.

Still confused, like I have some drawable icons. How can I place them in
the calendar
On Wed, May 3, 2017 at 9:30 PM SundeepK<EMAIL_ADDRESS>wrote:

This is a duplicate of #181
https://github.com/SundeepK/CompactCalendarView/issues/181. Have a read
through and let me know if you don't understand. Short version is, not
currently supported. If you want it, you can fork the code and the part
where the dot is drawn, you can just draw your icon instead.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SundeepK/CompactCalendarView/issues/193#issuecomment-298955832,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVB6ExQWxVBzk5O6NFh3tuT6WVHFIZU5ks5r2KSGgaJpZM4NPHeA
.

Hi, did you manage to solve this problem? Do you still need help with this? I can perhaps have a look at creating some sample code if you still need it.

It would be great if you can share some sample code
On Thu, 13 Jul 2017 at 3:10 AM, SundeepK<EMAIL_ADDRESS>wrote:

Hi, did you manage to solve this problem? Do you still need help with
this? I can perhaps have a look at creating some sample code if you still
need it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SundeepK/CompactCalendarView/issues/193#issuecomment-314905330,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVB6E44biALd54AlmgI8X1aYNxe66rXTks5sNT1RgaJpZM4NPHeA
.

Ok thanks, will try get something for you soon.

Have a look at issue #181.

thanks,
this worked for me :)

On 17-Jul-2017, at 4:15 AM, SundeepK<EMAIL_ADDRESS>wrote:
Have a look at issue #181 https://github.com/SundeepK/CompactCalendarView/issues/181.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/SundeepK/CompactCalendarView/issues/193#issuecomment-315643386, or mute the thread https://github.com/notifications/unsubscribe-auth/AVB6ExgC0_feYOiRCYa73R4fSU9oGeIWks5sOpJ0gaJpZM4NPHeA.


Improved log fetching performance

Parallelize fetching logs in EthereumAdapter
Increasing block range size again as soon as possible after getting past a block.

Potentially do the same for traces.

Point 1 is done. As I outlined in #914, imo the rest can be considered low priority.


Close date picker onBlur

The date picker should close when the corresponding input field loses focus.

I agree, date picker should close onBlur or onFocusOut or the component.

It's not possible to close it on blur because when the user clicks on the calendar, the input loses the focus and it would get closed.
You could try it yourselves with something like:
<DateTime ref="picker" inputProps={{ onBlur: () => this.refs.picker.closeCalendar() }} />

This actually covers my use case well. Im mainly concerned when there are multiple DateTime widgets on the screen in a single form and you quickly tab through the form and they all remain open.
I just implemented the above and it works well - however, the next/previous buttons are marked us as button elements so when tabbing the focus goes to these rather than the next field. Is there any particular reason these are buttons given that the rest of the widget doesn't seem to be keyboard accessible? These could be changed to spans.

For me the example did not work, I could not select any date anymore. This hack works for me
inputProps={{
                            onBlur: () => {
                                setTimeout(
                                    () => this.refs.picker.closeCalendar(),
                                    100
                                )
                            }
                      }}

@mleonard87 I think the <button> elements are replaced with <span> elements. I've tried having multiple components on a single page and I can successfully tab between them without anything else but the <input> element getting focus.
Thanks for the solution and comments. I will close this now, if anyone is still having issues just re-open this again.


Typescript error: nuxt.config.ts build.babel doesn't have presets in TransformOptions

Version
2.7.1
Reproduction link
https://github.com/nuxt/nuxt.js/blob/dev/packages/config/types/build.d.ts
Steps to reproduce
Typescript error: nuxt.config.ts build.babel doesn't have presets in TransformOptions
What is expected ?
export interface NuxtConfigurationBuild {
babel?: TransformOptions
}
I expected TransformOptions should have presets
What is actually happening?
const nuxtConfig: NuxtConfiguration = {
build: {
presets <====
}
}
presets which makes typescript error

This bug report is available on Nuxt community (#c9245)

@jackie-park
TransformOptions interface has presets property : https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/babel__core/index.d.ts#L254
You seem to do
build : {
  presets:
}

instead of
build: {
  babel: {
    presets:
  }
}

It was no problem with 2.6.x and works well in 2.7.x when I remove NuxtConfiguration type.
Theres's a test below in nuxt.js project.
https://github.com/nuxt/nuxt.js/blob/d811ef7fea697f43a75ef4fb93f5a9a706ec5d6d/packages/config/test/options.test.js#L212

@jackie-park I still don't get what's wrong, please provide exact configuration snippet that throw an error and please also provide this error.

@jackie-park Alright, this PR https://github.com/nuxt/nuxt.js/pull/5785 fixes the issue (when presets function is used) and will be available in next release.
In the meantime I think you should be able to put a // @ts-ignore comment above the presets so it doesn't throw an error.

@kevinmarrec Thank you 👍

Fix released in 2.8/0


applyDynamic does not support passing a vararg parameter

it will be nice if it did so that one can do this
js.Dynamic.literal(map.toSeq: _*)

Yea, that's a scalac bug, actually. You don't need js.Dynamic.literal per se. Any applyDynamic causes scalac to bug out. Please report this to the scalac JIRA tracker: https://issues.scala-lang.org/

Ok thanks. I reported here https://issues.scala-lang.org/browse/SI-9308


Improved model-meta-attribute checks

Currently the model-meta-attribute check can only check if an attribute is present.
However I have two additional type of checks that I require:

Being able to check that a meta attribute is not set (e.g. db_table)
Being able to check the value of a meta attribute, for instance if managed = False or managed = True

Any chance of adding support for this?

Any chance of adding support for this?

Hardly as standard checks but custom checks are planned (you will be able to write your own checks using this app infra).
That will expose currently internal API so it must be stabilized first, can't provide ETA at the moment.


Tailwind Not Found when deploying to Heroku

I am having challenge deploying my React App to Heroku. I used and declare Tailwindcss in my devDependencies. When I tried to deploy to Heroku, I got the error in the bash that Tailwind is not found.

Whithout seeing the log message, I can give you this guess.
Heroku runs on production environment so "devDependencies" are not installed.
To run tailwind commands you should put tailwindcss on the "Dependencies" part of your packages.json.

Expanding on my previous answer, here is what Heroku's documentation says

By default, Heroku will install all dependencies listed in package.json under dependencies and devDependencies.
After running the installation and build steps Heroku will strip out the packages declared under devDependencies before deploying the application.

Ok. Thanks. I'll look into it
On Mon, Jun 1, 2020, 11:09 PM Cleiton Floss<EMAIL_ADDRESS>wrote:

Expanding on my previous answer, here is what Heroku's documentation says
https://devcenter.heroku.com/articles/nodejs-support#package-installation
By default, Heroku will install all dependencies listed in package.json
under dependencies and devDependencies.
After running the installation and build steps Heroku will strip out the
packages declared under devDependencies before deploying the application.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tailwindcss/tailwindcss/issues/1771#issuecomment-637151586,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH2WWNTIYEXRERPLLKIGHK3RUQRJZANCNFSM4NAPMRHQ
.


[bug] Nonce service should be a singleton across Gateway

Description
Each Gateway service makes use of a NonceService class that utilizes Redis to keep track of submitted extrinsics & nonce usage, so that multiple extrinsic submissions in a short period of time don't fall afoul of either duplicate nonces or holes in the nonce.
Because each service uses the same keypair/seed phrase, the Redis key used for the NonceService should be common across the services; currently the service-specific key prefix applies to the nonce service keys as well, which makes it more likely that the services could encounter nonce problems.
Another potential solution would be to provision different keypairs for each service, to avoid cross-service nonce issues.

@JoeCap08055 prefix by pub key


Suggestions

Dependency Management Extras
PHP SemVer: https://github.com/vierbergenlars/php-semver
DataStructures
Tree: https://github.com/nicmart/Tree
Tries, Quads: https://github.com/MarkBaker
URL
URI: https://github.com/thephpleague/uri
HTTP
Diactoros: https://github.com/zendframework/zend-diactoros
Security
Roave Security: https://github.com/Roave/SecurityAdvisories
PHP Malware Finder: https://github.com/nbs-system/php-malware-finder
Queue
Queue: https://github.com/indigophp/queue
Gearman: http://gearman.org/
Celery PHP: https://github.com/gjedeer/celery-php
Event
Sabre Event: http://sabre.io/event/
Templates
Foil: http://www.foilphp.it/
Auth
Aura Auth: https://github.com/auraphp/Aura.Auth
Gatekeeper: https://github.com/psecio/gatekeeper
PHPoAuthLib: https://github.com/Lusitanian/PHPoAuthLib
DI
Ray Di: http://github.com/koriym/Ray.Di
DB
Analogue: https://github.com/analogueorm/analogue
Liquibase: http://www.liquibase.org/download/index.html
Config
PHP DotEnv: https://github.com/vlucas/phpdotenv
Symfony Config: http://symfony.com/doc/current/components/config/index.html
Debugging/Errors
BooBoo: https://github.com/thephpleague/booboo
Prophiler: http://prophiler.fabfuel.de/
NoSQL
Mandango: http://mandango.org/
Cache
Cache Tool: https://github.com/gordalina/cachetool
Web Applications
Web Securify: http://www.websecurify.com/
Cachet: https://github.com/cachethq/cachet
Micro Framework Extras
Config Service Provider: https://github.com/igorw/ConfigServiceProvider
Data Structures
TopSort: https://github.com/marcj/topsort.php
Deployment
PHPloy: https://github.com/banago/PHPloy
Testing
Factory Muffin: https://github.com/thephpleague/factory-muffin
Faker CLI: https://github.com/bit3/faker-cli
Phactory: https://github.com/chriskite/phactory
Scientific
LinearAlgebra: https://github.com/mcordingley/LinearAlgebra
Scientific: https://github.com/mcordingley/Scientific
Statistical Classifier: https://github.com/camspiers/statistical-classifier
Security
Version Scan: https://github.com/psecio/versionscan
Random Compat: https://github.com/paragonie/random_compat
File
Burgomaster: https://github.com/mtdowling/Burgomaster
Distill: https://github.com/raulfraile/distill
Extractor: https://github.com/mmoreram/extractor
Redaktilo: https://github.com/gnugat/redaktilo
Zipper: https://github.com/Chumper/Zipper
Cache
Cache: https://github.com/desarrolla2/Cache
Date
Business Days: https://github.com/iansltx/BusinessDays
I18N/L10N
Phred: https://github.com/nazariyg/Phred
Debugging
Patchwork Dumper: https://github.com/nicolas-grekas/Patchwork-Dumper
Markup
FluentDOM: https://github.com/FluentDOM/FluentDOM
XML String Streamer: https://github.com/prewk/xml-string-streamer
XML Faker: https://github.com/prewk/xml-faker
String
Redaktilo: http://gnugat.github.io/redaktilo/
Diff: https://github.com/sebastianbergmann/diff
Currency
Money: https://github.com/ulabox/money
Code Analysis
PDepend: http://pdepend.org/
Casting
PolyCast: https://github.com/theodorejb/PolyCast
Errors
Shit Happens: https://github.com/brandonsavage/shithappens
Image
Crop: https://github.com/stojg/crop
FaceDetect: http://www.xarg.org/project/php-facedetect/
Imanee: http://imanee.io/
Misc
PHPBackporter: https://github.com/nikic/PHP-Backporter
Sergeant: https://github.com/acairns/sergeant
PHP PM: https://github.com/marcj/php-pm
Instantiator: https://github.com/Ocramius/Instantiator
Gliph: https://github.com/sdboyer/gliph
Get In: https://github.com/igorw/get-in
Recoil: https://github.com/recoilphp/recoil
Ray Aop: https://github.com/koriym/Ray.Aop
PHP-PM: https://github.com/marcj/php-pm
Mondator: https://github.com/mandango/mondator
HamCrest: https://github.com/hamcrest/hamcrest-php
Transducers: https://github.com/mtdowling/transducers.php
File Get Contents: https://github.com/padraic/file_get_contents
Guzzle Promises: https://github.com/guzzle/promises
Tactician: http://tactician.thephpleague.com/
MessageBus: https://github.com/SimpleBus/MessageBus
Optimus: https://github.com/jenssegers/optimus
ValueObjects: https://github.com/nicolopignatelli/valueobjects
PHP Internals
PHP Language Specification: https://github.com/php/php-langspec
Caching
GeekCache: https://github.com/karptonite/geekcache
3rd Party APIs
Facebook SDK: https://github.com/facebook/facebook-php-sdk-v4
Youtube API: https://github.com/madcoda/php-youtube-api
Tumblr: https://github.com/tumblr/tumblr.php
YubiKey: https://github.com/enygma/yubikey
REST
RAML: https://github.com/alecsammon/php-raml-parser
Lionframe: https://github.com/Lakion/Lionframe
Social
PHP Twient: https://github.com/makotokw/php-twient
Backup
Backup Manager: https://github.com/heybigname/backup-manager
Search
SphinxSearch: https://github.com/ripaclub/sphinxsearch
CLI
Gush: https://github.com/gushphp/gush
PHP SS: https://github.com/Herzult/php-ssh
Web Apps
Pushman: https://github.com/PushmanPHP/pushman
Forum
Flarum: https://github.com/flarum/flarum
PHP 7
PHP 7 Compatibility Checker: https://github.com/sstalle/php7cc
Websites
Pipe: http://thepipe.io/
OAuthBible: http://oauthbible.com/
PHP Package Checklist: http://phppackagechecklist.com/
12 Factor App: http://12factor.net/
Dev Human: http://dev-human.com/
PHP Developer: http://www.phpdeveloper.org/
No Capes: http://nocapes.net/
PHP Annotated Monthly: http://blog.jetbrains.com/phpstorm/category/php-annotated-monthly/
Utilities
Polyfill: https://github.com/symfony/polyfill
Books
Domain Driven Design with PHP Examples: https://leanpub.com/ddd-in-php
Symfony Framework Deep Dive: https://leanpub.com/symfonyframeworkdeepdive-security
Essential JS Design Patterns: http://addyosmani.com/resources/essentialjsdesignpatterns/book/
Modern PHP: https://www.safaribooksonline.com/library/view/modern-php/9781491905173/
Other
Eloquent: https://github.com/eloquent
Mail-in-a-box: https://github.com/JoshData/mailinabox
Stellar (Git for Databases): https://github.com/fastmonkeys/stellar
Recki CT: https://github.com/google/recki-ct
Pixeler: https://github.com/lastguest/pixeler
Nightrain: https://github.com/naetech/nightrain
MessageBus: https://github.com/SimpleBus/MessageBus
PHPComplex: https://github.com/MarkBaker/PHPComplex
Libraries.io: http://libraries.io/
Articles
It's All About Time: http://blog.ircmaxell.com/2014/11/its-all-about-time.html
What About Garbage?: http://blog.ircmaxell.com/2014/12/what-about-garbage.html
When To Declare Classes Final: http://ocramius.github.io/blog/when-to-declare-classes-final/
ClearPHP: https://github.com/dseguy/clearPHP
Wisdon of the Elephant: http://wisdomoftheelephpant.com/
Securely Distributing PHARS: http://blog.astrumfutura.com/2015/03/securely-distributing-phars-pitfalls-and-solutions/
How to Safely Generate Random Strings and Integers in PHP: https://paragonie.com/blog/2015/07/how-safely-generate-random-strings-and-integers-in-php?from=hackernews
What Is PHP Doing?: http://derickrethans.nl/what-is-php-doing.html
Speeding Up Database Calls With PDO: http://www.dragonbe.com/2015/07/speeding-up-database-calls-with-pdo-and.html

The URL of Ray.Di and Ray.Aop are changed to https://github.com/ray-di/Ray.Di and https://github.com/ray-di/Ray.Aop. They up the version 2.

I believe following repo can be added in auth section. Please have a look. Seems to be an active project.
https://github.com/emreakay/CodeIgniter-Aauth

+1 :+1: :+1:

indigophp/queue is deprecated in favor of Bernard

I would like to suggest exakat: http://www.exaiat.io/ and https://github.com/exakat/exakat
This is a static code analyzer that checks whole projects for security, performances, code quality and PHP compliance. It provide over 500 analysis, all documented, and export results in various formats. It is unit tested, open source and is a mine of information about any PHP code, without even reading the latter.
Exakat has an ansible script for installation. It works best on PHP 7, but is compatible with PHP 5.3 to 7.1-dev and is able to check code with all those versions too. Composer installation is not yet ready.
I'll do a PR if this looks ok.

Website is not available.

Fixed the URL in the previous comment to www.exakat.io

@ziadoz closing this in favor of actual prs :)


The movements of the structures without bonds after Clean Up action.

Affected Version: Release/2.1
Steps to reproduce:
Scenario I

Click the “N” button on the Atom Palettebar and click the canvas.
Verify that NH3 molecule has appeared on the canvas.
Paste benzene from templates on another place of the canvas and distort it lightly.
Click the Clean Up button.

Actual result
The benzene structure has become undistorted and has not changed the position. The NH3 structure has moved to the left upper corner of the canvas.
Expected result
The benzene structure should become undistorted. The positions of the structures should not be changed.
Note: Similar behavior is observed for any case when any structure without bonds and any other structure with bonds take place on the canvas.
Scenario II

Click the “N” button on the Atom Palettebar and click the canvas.
Verify that NH3 molecule has appeared on the canvas.
Repeat step 1 with different atoms to create different structures without bonds.
Create any structure with 2 or more bonds, for example a chain with 2 bonds.
Click the Clean Up button.

Actual result
All structures without bonds have overlapped each other in the left upper corner of the canvas.
Expected result
The positions of the structures should not be changed.

Releted issue: https://github.com/epam/Indigo/issues/426


Check results of joining to the cluster

Joining cluster may fail, so we need to handle failure according to autocluster_failure setting.
Before https://github.com/rabbitmq/rabbitmq-server/pull/868 even {ok, already_member} should be treated as an error, but I'm not so sure about this.

[Current coverage][cc-pull] is 36.95%

Merging [#72][cc-pull] into [master][cc-base-branch] will not change coverage

@@             master        #72   diff @@
==========================================
  Files            12         12
  Lines           460        460
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            170        170
  Misses          290        290
  Partials          0          0

Powered by Codecov. Last updated by [3f0b7aa...07e1871][cc-compare]
[cc-base-branch]: https://codecov.io/gh/aweber/rabbitmq-autocluster/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/aweber/rabbitmq-autocluster/compare/3f0b7aa7d70cf4c0a25ea1819d3e0afd01ec9c91...07e187147085ea7b5f3ebaddc023d2a855e129ed
[cc-pull]: https://codecov.io/gh/aweber/rabbitmq-autocluster/pull/72?src=pr

That's awesome, thanks for the contribution. Would have gotten bonus points for adding test coverage, but it's still very appreciated.


fix(package): update mathjs to version 3.10.3

Closes #36
https://greenkeeper.io/

Coverage remained the same at 73.394% when pulling e310b2bfd44c6bcb0c8d3b92cba54ed73203ccc6 on greenkeeper/mathjs-3.10.3 into 97df512b1bc01b39aad38dc3f628a924fc7e75d3 on master.


Details tooltip

DetailsTooltip component
Use SDS components instead of local
Ignoring some TypeScript errors until SDS is updated (will do it after all changes are good locally)

Preview is available here:http://stellar-demo-wallet-pr220.previews.kube001.services.stellar-ops.com/

Preview is available here:http://stellar-demo-wallet-pr220.previews.kube001.services.stellar-ops.com/


Can fall through one way platforms

Windows 10
Godot 3.0 6b99950
Can fall through platforms with one way collision enabled.
Expect not to fall through them.
Start jumping and moving left to right several times and you will fall through.
Press arrow Up to jump arrow left/right to move
Attached is animated gif showing what happens. Also in imgur link of the same thing. Also project file.
https://i.imgur.com/WG9RUHL.gifv

OneWayCollisionProblems.zip

CC @RandomShaper

In 3.0 the OWC implementation is @reduz's, IIRC. Anyway, I could try to debug it, but this week I'm out of town.

I can't reproduce the bug (on linux), does the same with other kind of shapes or with capsule only?

I think the "bug" gets triggered when you touch another platform. Here I changed it to a square and enabled collision shapes in the debugger. As soon as I touch the other platform I fall.

With the capsule collider I don't fall as soon as I touch it but when I walk back to the left after jumping.
Very strange. I can try to strip out more of the project if you think that will help debug?

Tested with rigidbodies too, these fail on different situations (like fall on rotation, hit on sides), so looks like 1 side shapes are badly broken now.

Rotated KinematicBody2D's shapes fail to hit 1 side shapes too.

Cool so im not crazy. :)
Anything I can do to help debug?

Please make a testcase!
On Sep 4, 2017 9:23 PM, "William Taylor"<EMAIL_ADDRESS>wrote:

Cool so im not crazy. :)
Anything I can do to help debug?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/10971#issuecomment-327042448,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z219w6-U906fJ59dCJ5vrpSTKNCmlks5sfJRpgaJpZM4PMNFN
.

Would be happy to if I can figure out how.

Ah, wait, @willt, you use rays to check collision and not full velocity on move_and_slide.
Rays do not work correctly on 1 side and may be introducing some extra bugs on your code...
move_and_slide uses full velocity on the method, you should not use delta time on that.

I've reduced the project even smaller. It doesn't uses rays at all now. No input etc. It doesn't trigger every time so run it a couple times if it doesn't work.

OneWayCollisionProblemsMin.zip

Your last project does vel*delta, that is multiplied by step internally and looks like the extreme low value introduces some bugs (related to the separation?).

The same logic is used in the demo here: https://github.com/godotengine/godot-demo-projects/blob/194655c80571595c67b11e9347b1d55316119e39/2d/kinematic_character/player.gd
If you don't have delta applied you would fly across the screen i think

that demo uses "force" to determine velocity, is not the same.

Just tested this code. Does the same thing.
And in that code on line 62 they have this "velocity += force*delta"
func _fixed_process(delta): 	var force = Vector2(0, 10) 	force.x += 2 	velocity += force 	velocity = move_and_slide(velocity, Vector2(0, -1))

Its called force in the demo but I don't think it's really force in the sense of force = mass * acceleration is it?
I believe the code I had was essentially the same except I was adding in acceleration.
At any rate without debating the laws of motion. I think its safe to say the bug happens with or without
delta being given to move_and_slide. Would you agree?

Yes, it happens at very low motion values.

Well, I can't seem to be able to make it fail anymore with the example projects, I will keep testing...

@eon-s Did you change something to make it stop failing?

Yes, but then tried on a clean project (2nd one) too, weird (rotated shapes keep failing, though).
I'll see if I can start some kind of testbed for kinematic bodies this week to make it easier to break them.

@eon-s
I have a suspicion that this commit https://github.com/godotengine/godot/commit/77ec304966436405e2975f7da480846cc074f838
is possibly when things broke. Downloading a build from here: https://bintray.com/calinou/godot/editor/2017-08-30.8ed1a28#files
I can not replicate the bug.

confirmed in my original project the bug does not exist.
Also I just backed out commit 77ec304966436405e2975f7da480846cc074f838 recompiled and problem is gone. I could try to debug further but I think @reduz will probably have better luck than me. :)
Let me know if you need anything else.

@RandomShaper did a better implementation of this for 2.1, so I guess I'll wait until he's back from vacation to improve this

Oh, it's been a week of work in Madrid. I've enjoyed my new company, but I'd not call it vacation. :)
Maybe this weekend I can check it.

Okay I think i've narrowed this down a bit more.
The problem seems to be with collision shapes that have been duplicated.
Please see attached gif and project. In project enable collision shapes in debug.
The middle platform and top right platform are duplicates of each other. (ie I have right clicked on one and clicked duplicate and dragged it over)
You will see if you are touching both at the same time you fall through.
Now that I have a better idea of whats going on ill poke around and see if I can resolve it but im not familiar with the code at all so im sure you will be able to fix it right away. :)

DuplicateCollisionBug.zip

oh i see, makes total sense that this makes it fail with the code I added,
i will work on it asap
On Sat, Sep 9, 2017 at 2:52 AM, William Taylor<EMAIL_ADDRESS>wrote:

Okay I think i've narrowed this down a bit more.
The problem seems to be with collision shapes that have been duplicated.
Please see attached gif and project. In project enable collision shapes in
debug.
The middle platform and top right platform are duplicates of each other.
(ie I have right clicked on one and clicked duplicate and dragged it over)
You will see if you are touching both at the same time you fall through.
Now that I have a better idea of whats going on ill poke around and see if
I can resolve it but im not familiar with the code at all so im sure you
will be able to fix it right away. :)
[image: 2017-09-08_22-46-46]
https://user-images.githubusercontent.com/340016/30237498-361ff436-94e8-11e7-8ea6-b7f1f0f25155.gif
DuplicateCollisionBug.zip
https://github.com/godotengine/godot/files/1289577/DuplicateCollisionBug.zip
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/10971#issuecomment-328256797,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z2x1uFTz9TgEUCGD_-Pf-VhF0Vngbks5sgievgaJpZM4PMNFN
.

confirmed fixed thanks!

I can confirm what @jahd2602 said. It is occurring with a circle/capsule container.


add Degraded condition to HostedCluster

What this PR does / why we need it:
Follow up to #1560
https://issues.redhat.com/browse/HOSTEDCP-495
Copies the Degraded condition on the HCP up to the user-facing HC.
As of this PR, the HC controller does not have any additional logic to observe degraded conditions within its management domain but that is in the plan e.g. deployment availability checks on CAPI and the CPO, etc
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

/lgtm


Detect support for --headless and use if available

Chrome 59 gained support for --headless.
This PR makes the runner use this if Headless is supplied to the
runner, if it's available, falling back to the old method
(headless_shell) if it is unavailable.
Unfortunately it does not seem possible to get chrome to list which
command line options it supports. I could not find a way of detecting
the feature, so we instead just use a check on the version number, which
is not the preferred solution but does the job.

For me, the above patch makes the tests pass without having anything other than chrome installed (no headless_shell).

Coverage increased (+0.05%) to 67.175% when pulling 67ed3e7c71eeddc38215112def3ba6fb3c8c51f8 on pwaller:native-headless into 4cc4ba1ca23fb669e5ec66cda5c2bcdc3c10f672 on knq:master.

Coverage increased (+0.4%) to 67.501% when pulling 67ed3e7c71eeddc38215112def3ba6fb3c8c51f8 on pwaller:native-headless into 4cc4ba1ca23fb669e5ec66cda5c2bcdc3c10f672 on knq:master.

Coverage increased (+0.4%) to 67.501% when pulling 67ed3e7c71eeddc38215112def3ba6fb3c8c51f8 on pwaller:native-headless into 4cc4ba1ca23fb669e5ec66cda5c2bcdc3c10f672 on knq:master.

I do not understand the test failures - they don't seem to be related to my change?

I'm not sure what the benefit of this would be? I believe Chrome ignores any invalid command line parameters, so if you pass --headless and expect it to work and it doesn't, then you're on a version of Chrome / browser that doesn't support --headless.

On another note, I'd like to keep "magic" to a minimum. Meaning, I don't think chromedp should be responsible for too much management of the browser process. There's a certain amount of invocation voodoo going on with the runner and client packages, for sure, but that's kind of the base minimum needed to "make it go." My concern with adding the additional logic in the command line option, is that there is likely a performance hit from exec.LookPath being called every time you launch a new --headless browser.

I'm not sure what the benefit of this would be?

My main motivation is to make the tests work without having to install headless_shell.

I believe Chrome ignores any invalid command line parameters, so if you pass --headless and expect it to work and it doesn't, then you're on a version of Chrome / browser that doesn't support --headless.

Except you then got a non-headless browser.

On another note, I'd like to keep "magic" to a minimum.

Understandable.

My concern with adding the additional logic in the command line option, is that there is likely a performance hit from exec.LookPath being called every time you launch a new --headless browser.

I understand where you're coming from, but I would expect this performance hit to be approximately nothing, given reasonable values for $PATH, and the "cost" of failed lookups would be quickly amortised by the OS's caches.
Also, my patch doesn't introduce any more lookpaths than were already present.

Coverage decreased (-0.05%) to 68.362% when pulling 368eaf4d0ca590e4d9f0ba7e23da794cd37893a8 on pwaller:native-headless into ccdd204500a4787c6496320d2906f708364973f6 on knq:master.

@pwaller I guess this would be the treatment for my issue: https://github.com/knq/chromedp/issues/109
Good job


Misunderstood self connection?

I've been looking for a way of turning on wireless debugging while outside of normal # WiFi connection spaces. So that I could activate Shizuku without WiFi. https://github.com/RikkaApps/Shizuku
I think that perhaps this project doesn't help with what I was trying as the device running the tetherfi networking can't itself connect to the network even though WiFi isn't turned off when this 'hotspot' is tured on?

Yeah, the hosting device cannot connect to the wifi direct network. A root app may be able to access hostapd directly, but at that point you wouldn't need Shizuku either


Remove references to removed fonts

The fonts were removed in #431, but the references weren't removed

Thanks Jake


Mobile Key: Unable to add Mobile Key.

https://github.com/bithyve/bitcoin-keeper/assets/62699947/a53e4a03-110d-416c-85e1-25a072131da0

When we go to add the mobile key as a signer we get this error.

Verified this issue on keeper app v 1.1.12(306)


Change token delimiter

Resolves #132

Codecov Report

Merging #167 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master     #167   +/-   ##
=========================================
  Coverage     81.93%   81.93%
  Complexity      980      980
=========================================
  Files           144      144
  Lines          2922     2922
  Branches        343      343
=========================================
  Hits           2394     2394
  Misses          443      443
  Partials         85       85

Impacted Files
Coverage Δ
Complexity Δ

...rc/main/java/tutorspet/logic/parser/CliSyntax.java
93.33% <100.00%> (ø)
1.00 <1.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ba5f6d...b051467. Read the comment docs.


2015 Year archive

Hi I was wondering when the 2015 year archive was coming out. Thanks! Happy to help deploy it if it makes sense

See https://github.com/igrigorik/githubarchive.org/issues/129 :)

cheers! thank you


Dataset improvements

Most of the time the dataset is implemented for ArrayBase<D, Ix2> for records and ArrayBase<D, Ix1> for targets. We could simplify a lot of impls by:

[x] renaming Dataset to DatasetBase
[x] implement Dataset and DatasetView
[ ] implement Dataframe
[x] support multi-targets and label probabilities

Implement for new types

[x] k-folding
[x] non-copying k-folding iter_fold
[x] bootstrapping
[x] shuffling

Improve ergonomics:

[x] feature naming
[ ] target naming

Hi, I would love to help with this, but I can't say that I understand completely what you have in mind. Could you explain it a little more? Is the following what you had in mind?

Keep what's already implemented for Dataset but rename the struct DatasetBase (kind of like ndarray with Array and ArrayBase?)
Add new types Dataset and DatasetView that use ArrayBase and ArrayView for the records and targets like you described. Would one do this by pub type Dataset = DatasetBase<ArrayBase<D, Ix2>,ArrayBase<D, Ix1>> ?

both points are right, we would just follow the ndarray model more closely. Because of the coming holidays, my plan was to schedule that for beginning of next year. You can give it a try, though it's rather diligence work as all algorithms have to be updated :D

Well I'll try then and I'll let you know how it goes 👍🏻

Hi, is there some specific reason why some datasets in the datasets crate are returned using a Vec for the targets instead of an Array1?

no this should default to Array1

I have one doubt about iter_fold. Shouldn't iterator items possibly outlive the object from which the iterator is created? I mean, one would have to:

take a mutable reference to the original dataset
split that reference into mutable slices
for each item in the iterator swap the content of some slices and then recombine the slices in two arrays

But at that point, the content of the various items would overlap, and calling collect on that iterator would yield a mess right? I mean if one wanted to use two items from the iterator at the same time then there would be a conflict on the contents of the slices composing the two items right?
Wouldn't such implementation need something like this proposal?
I'm asking because as a rust noob I just can't wrap my head around it

I have one doubt about iter_fold. Shouldn't iterator items possibly outlive the object from which the iterator is created? I mean, one would have to:

take a mutable reference to the original dataset
split that reference into mutable slices
for each item in the iterator swap the content of some slices and then recombine the slices in two arrays

But at that point, the content of the various items would overlap, and calling collect on that iterator would yield a mess right? I mean if one wanted to use two items from the iterator at the same time then there would be a conflict on the contents of the slices composing the two items right?
Wouldn't such implementation need something like this proposal?
I'm asking because as a rust noob I just can't wrap my head around it

We can't ensure that somebody is holding a reference while modifying the content. One solution would involve RefCell, which basically moves the borrow check to the runtime and panics when the content is atm borrowed for mutation.
Another solution would drop the iterator completely and just take a closure with Dataset -> Model. The caller would then ensure that the reference to the dataset is dropped after the model is trained.

We can't ensure that somebody is holding a reference while modifying the content. One solution would involve RefCell, which basically moves the borrow check to the runtime and panics when the content is atm borrowed for mutation.
Another solution would drop the iterator completely and just take a closure with Dataset -> Model. The caller would then ensure that the reference to the dataset is dropped after the model is trained.

And what would a function like the second one you mention return? The k validation scores or the k couples (model, validation set)? I'm trying to make some sort of connection with scikit-learn's cross_val_score.

And what would a function like the second one you mention return? The k validation scores or the k couples (model, validation set)? I'm trying to make some sort of connection with scikit-learn's cross_val_score.

It should return impl Iterator<Item = (model, validation set)>, but I don't know wether the model type can be chosen on the return type of a closure

It should return impl Iterator<Item = (model, validation set)>, but I don't know wether the model type can be chosen on the return type of a closure

Maybe the Transformer, Fit and Predict traits could be used to that end?

Maybe the Transformer, Fit and Predict traits could be used to that end?

Ok so I thought I had it because now it complies but there's one tiny problem. I tried to implement iter_fold for fittable algorithms but the problem here is that fit requires that the dataset used for fitting has the same lifetime as the model. That means that the view that is given as input to the closure can't be used to train the model because it's too short-lived.

Ok so I thought I had it because now it complies but there's one tiny problem. I tried to implement iter_fold for fittable algorithms but the problem here is that fit requires that the dataset used for fitting has the same lifetime as the model. That means that the view that is given as input to the closure can't be used to train the model because it's too short-lived.

mh this was introduced to associate the correct lifetime for the kernel matrix here. We could copy the kernel matrix and avoid the lifetime there

mh this was introduced to associate the correct lifetime for the kernel matrix here. We could copy the kernel matrix and avoid the lifetime there

I've been a little busy lately and I'm proceeding slowly so I thought I'd give a little "status-report" here, also to get a second opinion on what I've been doing.
First part:

removed lifetime requirement on the training set for fittable models;
completed the implementation of iter_fold for fittable models;
tested the implementation on a mock struct implementing Fit;

Second part:
Here things get a little less straightforward. I didn't want to force people using SVMs to clone their kernel every time, so I am trying to change SVMs to accept either an owned kernel or a "view" of one. To that end I changed linfa-kernel in this way:

Added two new enum variants to KernelInner to be able to take a view of sparse or dense kernel matrices;
Renamed Kernel to KernelBase in linfa-kernel;
Added types:

KernelArray : uses ArrayBase<D, Ix2> as records
Kernel: uses Array2 as records
Kerneliew: uses ArrayView2 as records

Moved implementations relying only on the inner matrix to KernelBase and the ones relying also on the dataset to KernelArray;
Implemented new methods for Kernel and KernelView where the first one takes ownership of the input data and both use owned inner matrices;
Implemented view methods for Kernel and KernelView where the first one takes a view of the dataset and both take views of their inner matrices;

This way instead of taking a reference to the kernel one can call kernel.view() and not worry about data being cloned.
Now I still need to modify linfa-svm to adjust it to these changes, but I plan to do it sometime in the next weeks. One thing that bothers me is: since the KernelInner can contain both a view or owned data I'm forced to specify a lifetime for the type even if I am not planning to use the view type. For example, the type Kernel still needs a lifetime even if it can't have a view as an inner matrix, but I don't know if that's avoidable somehow.

I've been a little busy lately and I'm proceeding slowly so I thought I'd give a little "status-report" here, also to get a second opinion on what I've been doing.
First part:

removed lifetime requirement on the training set for fittable models;
completed the implementation of iter_fold for fittable models;
tested the implementation on a mock struct implementing Fit;

Second part:
Here things get a little less straightforward. I didn't want to force people using SVMs to clone their kernel every time, so I am trying to change SVMs to accept either an owned kernel or a "view" of one. To that end I changed linfa-kernel in this way:

Added two new enum variants to KernelInner to be able to take a view of sparse or dense kernel matrices;
Renamed Kernel to KernelBase in linfa-kernel;
Added types:

KernelArray : uses ArrayBase<D, Ix2> as records
Kernel: uses Array2 as records
Kerneliew: uses ArrayView2 as records

Moved implementations relying only on the inner matrix to KernelBase and the ones relying also on the dataset to KernelArray;
Implemented new methods for Kernel and KernelView where the first one takes ownership of the input data and both use owned inner matrices;
Implemented view methods for Kernel and KernelView where the first one takes a view of the dataset and both take views of their inner matrices;

This way instead of taking a reference to the kernel one can call kernel.view() and not worry about data being cloned.
Now I still need to modify linfa-svm to adjust it to these changes, but I plan to do it sometime in the next weeks. One thing that bothers me is: since the KernelInner can contain both a view or owned data I'm forced to specify a lifetime for the type even if I am not planning to use the view type. For example, the type Kernel still needs a lifetime even if it can't have a view as an inner matrix, but I don't know if that's avoidable somehow.

I've been a little busy lately and I'm proceeding slowly so I thought I'd give a little "status-report" here, also to get a second opinion on what I've been doing.

I'm currently re-locating to a different city and this skipped completely my radar, sorry for the late response.

First part:
* removed lifetime requirement on the training set for fittable models;

* completed the implementation of `iter_fold` for fittable models;

* tested the implementation on a mock struct implementing `Fit`;

:+1:

Second part:
Here things get a little less straightforward. I didn't want to force people using SVMs to clone their kernel every time, so I am trying to change SVMs to accept either an owned kernel or a "view" of one. To that end I changed linfa-kernel in this way:
* Added two new enum variants to `KernelInner` to be able to take a view of sparse or dense kernel matrices;

* Renamed `Kernel` to `KernelBase` in `linfa-kernel`;

* Added types:

  * `KernelArray` : uses `ArrayBase<D, Ix2>` as records
  * `Kernel`: uses `Array2` as records
  * `Kerneliew`: uses `ArrayView2` as records

* Moved implementations relying only on the inner matrix to `KernelBase` and the ones relying also on the dataset to `KernelArray`;

* Implemented `new` methods for `Kernel` and `KernelView` where the first one takes ownership of the input data and both use owned inner matrices;

* Implemented `view` methods for `Kernel` and `KernelView` where the first one takes a view of the dataset and both take views of their inner matrices;

This way instead of taking a reference to the kernel one can call kernel.view() and not worry about data being cloned.

this seems to be the straightforward way to have a view on a kernel matrix.

At this point the problem is that I would need to be able to differentiate between an SVM that has a view of data that lives as long as itself and one that owns the data in itself. I don't know if that can be done with just one type, I'm trying not to complicate the code too much.

you are here in a similar spot as the std::borrow::Cow implementation because you cannot suppress the lifetime in the type signature

Now I still need to modify linfa-svm to adjust it to these changes, but I plan to do it sometime in the next weeks. One thing that bothers me is: since the KernelInner can contain both a view or owned data I'm forced to specify a lifetime for the type even if I am not planning to use the view type. For example, the type Kernel still needs a lifetime even if it can't have a view as an inner matrix, but I don't know if that's avoidable somehow.

no you cannot avoid this without introducing SvmView and Svm, but then you have to decide what to return in SvmParams::fit.
I think that it would be reasonable to copy only those sample points which are support vectors. The whole point of SVM is to find a few SV's which can describe the separating hyperplane and we can easily copy those. (btw. this is only the case for the non-linear case, otherwise we do not need to cache the samples and have memory complexity O(n)) The following changes would be reasonable:

introduce a KernelInner::Dynamic: a placeholder where the distance function is calculated on-the-fly without caching
add a Kernel::select function: this takes the indices of the sub-samples and returns a copy of the kernel with owned records and empty (i.e. KernelInner::Dynamic) kernel matrix.
store only the smaller kernel of the support vectors Svm and don't borrow the whole kernel beyond the fitting process
in Svm::predict the weighted_sum is only passed the actual scaling factors of the SV solution (we could also think about not storing the kernel at all for the linear case, but would probably rename the Kernel::select then)

hope this is plausible, I think we should not make the API more complex as it already is

I've been a little busy lately and I'm proceeding slowly so I thought I'd give a little "status-report" here, also to get a second opinion on what I've been doing.

I'm currently re-locating to a different city and this skipped completely my radar, sorry for the late response.

First part:
* removed lifetime requirement on the training set for fittable models;

* completed the implementation of `iter_fold` for fittable models;

* tested the implementation on a mock struct implementing `Fit`;

:+1:

Second part:
Here things get a little less straightforward. I didn't want to force people using SVMs to clone their kernel every time, so I am trying to change SVMs to accept either an owned kernel or a "view" of one. To that end I changed linfa-kernel in this way:
* Added two new enum variants to `KernelInner` to be able to take a view of sparse or dense kernel matrices;

* Renamed `Kernel` to `KernelBase` in `linfa-kernel`;

* Added types:

  * `KernelArray` : uses `ArrayBase<D, Ix2>` as records
  * `Kernel`: uses `Array2` as records
  * `Kerneliew`: uses `ArrayView2` as records

* Moved implementations relying only on the inner matrix to `KernelBase` and the ones relying also on the dataset to `KernelArray`;

* Implemented `new` methods for `Kernel` and `KernelView` where the first one takes ownership of the input data and both use owned inner matrices;

* Implemented `view` methods for `Kernel` and `KernelView` where the first one takes a view of the dataset and both take views of their inner matrices;

This way instead of taking a reference to the kernel one can call kernel.view() and not worry about data being cloned.

this seems to be the straightforward way to have a view on a kernel matrix.

At this point the problem is that I would need to be able to differentiate between an SVM that has a view of data that lives as long as itself and one that owns the data in itself. I don't know if that can be done with just one type, I'm trying not to complicate the code too much.

you are here in a similar spot as the std::borrow::Cow implementation because you cannot suppress the lifetime in the type signature

Now I still need to modify linfa-svm to adjust it to these changes, but I plan to do it sometime in the next weeks. One thing that bothers me is: since the KernelInner can contain both a view or owned data I'm forced to specify a lifetime for the type even if I am not planning to use the view type. For example, the type Kernel still needs a lifetime even if it can't have a view as an inner matrix, but I don't know if that's avoidable somehow.

no you cannot avoid this without introducing SvmView and Svm, but then you have to decide what to return in SvmParams::fit.
I think that it would be reasonable to copy only those sample points which are support vectors. The whole point of SVM is to find a few SV's which can describe the separating hyperplane and we can easily copy those. (btw. this is only the case for the non-linear case, otherwise we do not need to cache the samples and have memory complexity O(n)) The following changes would be reasonable:

introduce a KernelInner::Dynamic: a placeholder where the distance function is calculated on-the-fly without caching
add a Kernel::select function: this takes the indices of the sub-samples and returns a copy of the kernel with owned records and empty (i.e. KernelInner::Dynamic) kernel matrix.
store only the smaller kernel of the support vectors Svm and don't borrow the whole kernel beyond the fitting process
in Svm::predict the weighted_sum is only passed the actual scaling factors of the SV solution (we could also think about not storing the kernel at all for the linear case, but would probably rename the Kernel::select then)

hope this is plausible, I think we should not make the API more complex as it already is

you are here in a similar spot as the std::borrow::Cow implementation because you cannot suppress the lifetime in the type signature

One way I've found is to define a trait "Inner" with all methods concerning only inner matrices and implement it for Array2, ArrayView2, CsMat, and CsMatView. Then I define an enum:
pub enum KernelInnerBase<K1: Inner, K2:Inner> {
    Dense(K1),
    Sparse(k2),
}

Now the definition of kernel becomes:
pub struct  Kernel<R: Records, K1: Inner, K2: Inner> {
    ....
    inner: KernelInner<K1,K2>,
    ....
}

So I can define Kernel<F> = Kernel<Array2<F>, Array2<F>, CsMat<F>>. Then the naming gets a little confusing because I think both KernelArrayView<ArrayView2<'a, F>, Array2<F>, CsMat<F>> and KernelView<ArrayView2<'a,F>, ArrayView2<'a,F>, CsMatView<'a,F>> would be useful.
With this kind of implementation, all methods like diag and such become just calls to the corresponding methods of either K1 or K2 depending on whether the inner matrix is sparse or not.
I agree that the API should stay as simple as possible. Right now I'm deep into studying/taking my finals so I'll slowly get to the points you made about SVM in the next weeks. 👍🏻

you are here in a similar spot as the std::borrow::Cow implementation because you cannot suppress the lifetime in the type signature

One way I've found is to define a trait "Inner" with all methods concerning only inner matrices and implement it for Array2, ArrayView2, CsMat, and CsMatView. Then I define an enum:
pub enum KernelInnerBase<K1: Inner, K2:Inner> {
    Dense(K1),
    Sparse(k2),
}

Now the definition of kernel becomes:
pub struct  Kernel<R: Records, K1: Inner, K2: Inner> {
    ....
    inner: KernelInner<K1,K2>,
    ....
}

So I can define Kernel<F> = Kernel<Array2<F>, Array2<F>, CsMat<F>>. Then the naming gets a little confusing because I think both KernelArrayView<ArrayView2<'a, F>, Array2<F>, CsMat<F>> and KernelView<ArrayView2<'a,F>, ArrayView2<'a,F>, CsMatView<'a,F>> would be useful.
With this kind of implementation, all methods like diag and such become just calls to the corresponding methods of either K1 or K2 depending on whether the inner matrix is sparse or not.
I agree that the API should stay as simple as possible. Right now I'm deep into studying/taking my finals so I'll slowly get to the points you made about SVM in the next weeks. 👍🏻

What if in SVMs the kernel was specified as a parameter? I mean something like this:
Svm::params().with_kernel(Kernel::params()).fit(&dataset)

That way fit would take the original dataset instead of the transformed kernel and inside fit one could do something like this:
let kernel_matrix = self.kernel.transform(dataset);
...
svm.kernel = kernel_matrix;
svm.dataset = dataset;
...

without the need to store dataset inside the kernel. Then the Svm could be improved to pick the support vectors from dataset and clone only them instead of the whole set.

What if in SVMs the kernel was specified as a parameter? I mean something like this:
Svm::params().with_kernel(Kernel::params()).fit(&dataset)

this should be called with_kernel_params or take the actual kernel matrix, alternatively we could add separate functions for the different kernel functions, like .with_gaussian(eps) etc.

That way fit would take the original dataset instead of the transformed kernel and inside fit one could do something like this:
let kernel_matrix = self.kernel.transform(dataset);
...
svm.kernel = kernel_matrix;
svm.dataset = dataset;
...

without the need to store dataset inside the kernel. Then the Svm could be improved to pick the support vectors from dataset and clone only them instead of the whole set.

sounds like a good solution :+1:

in Elements of Statistical Learning the training vectors are sometimes called predictors, perhaps that's a better name than records

To me, calling dataset.records() sounds more straightforward than dataset.predictors(), just from what I would generally expect from a dataset-like structure, but since this structure is focused purely on ML maybe it would make sense?. I know that Understanding Machine Learning uses predictor to refer to the hypothesis learned and instances to refer to domain points, so I think that predictors could be misleading to people who learned ML through that book. Other names that I heard a lot in my uni courses are samples or examples but I don't know how often any of those terms are actually used in literature.

Good point! ESL uses predictors as a synonym for training data in the Least Angle Regression section and it should only be used for those features which are used in the prediction. If you have selected a subset to decrease the variance of your model, then only those are the predictors as opposed to records. I don't like samples, examples because they can refer to the records as well as targets. I think we should stick to records

I agree, even if it may not be the best possible name I think that it is general enough to at least avoid conflicts 👍🏻

I've run into a problem when trying to implement the proposed changes to the regression metrics traits.
Since both SingleTargetRegression and MultiTargetRegression provide methods with the same name, as soon as I implement both SingleTargetRegression and MultiTargetRegressionfor a certain type I get errors because there are multiple methods available with the same name, even if they apply to different compared targets.
I don't know if there is a nice way to solve this, one solution is to keep defaulting to multi-target as it happens now or either to make the API a little uglier and rename them to single_target_metric and multi_target_metric.
If I use associated types it gets even worse, because I can only provide one implementation (and so only one option for the associated type) for each AsTarget type, since I can't take the associated type as a template parameter in the trait definition.
I think that the problem can't be overcome in rust since there is no concept of function overloading.
This is what I've tried so far:
Case 1: no associated types
pub trait SingleTargetRegression<F,T> : AsTargets<F> {
    fn max_error(&self, &T) -> ... {...}
}

pub trait MultiTargetRegression<F,T> : AsTargets<F> {
    fn max_error(&self, &T) -> ... {...}
}

impl<F,type1> SingleTargetRegression<F,type1> for type {}
impl<F,type2> MultiTargetRegression<F,type2> for type {}

// error: multiple methods with the same name
type.max_error(..)

This means that I can't have Array2 use single target metrics when compared to an array1 and multi-target metrics when compared to array2
Case 2: Associated types
Defining type in trait template
pub trait SingleTargetRegression<F,T> : AsTargets<F> {
    // error: associated type defaults are unstable
    type Tar = T;
    fn max_error(&self, &T) -> ... {...}
}

Defining type in impl template
pub trait SingleTargetRegression<F> : AsTargets<F> {
    type Tar : AsTargets<F>;
    fn max_error(&self, &T) -> ... {...}
}

// error: unconstrained type parameter
impl<type2> SingleTargetRegression<F> for type {
    type Tar = type2;
}

implementing twice
pub trait SingleTargetRegression<F> : AsTargets<F> {
    type Tar : AsTargets<F>;
    fn max_error(&self, &T) -> ... {...}
}

// error: conflicting implementation
impl SingleTargetRegression<F> for type {
    type Tar = base_type;
}

impl SingleTargetRegression<F> for type {
    type Tar = base_type2;
}

mh I wasn't aware of the Array2/Array1 implementation, but yeah I think it is not possible with the current trait system (though I'm absolutely not an expert there) the only workaround I see is to implement MultiTargetRegression like this:
impl<F: Float, D: Data<Elem = F>> MultiTargetRegression<F, ArrayBase<D, Ix1>> for ArrayBase<D, Ix2> {}

and hope that most people are using predict in combination with this trait.

Yes it is implemented like that, just allowing the two arrays to have different data, the only issue is that array2 can only implement multi target metrics.
(In this comment and in the ones before i am using array`array2`, loosely to indicate both owned arrays and views)

I think that .with_feature_names can provide a longer vector than there are features.


.buildASDFSystem: allow overloading wrapLisp defaults

Description of changes
Change the order of precedence for buildASDFSystem to allow systems to override lisp flags for their build. This is needed, I think, to allow for building an executable with "sbcl --script" rather than the default flags.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@Uthar you wrote nix-cl.nix, right? Can you weigh in on whether this is a safe change/whether there's a better way to override the interpreter flags? Thanks in advance.

To override flags, I was using the flags parameter of wrapLisp:
pkgs.wrapLisp { pkg = sbcl; faslExt = "fasl"; flags = ["--dynamic-space-size" "4096"];

(just re-wrap the previous sbcl with new flags)

FWIW the Nix build of SBCL now supports a NIX_SBCL_DYNAMIC_SPACE_SIZE envvar. It's not a general fix but it should help with your particular case.

Gotcha, closing this then.


Remove deprecated methods

[Current coverage][cc-pull] is 99.82%

Merging [#87][cc-pull] into [master][cc-base-branch] will increase coverage by 1.06%

2 files in ...in/scala/io/iteratee were modified. more
2 files (not in diff) in ...io/iteratee/internal were modified. more

@@             master        #87   diff @@
==========================================
  Files            20         20
  Lines           565        559     -6
  Methods         550        544     -6
  Messages          0          0
  Branches         15         15
==========================================
  Hits            558        558
+ Misses            7          1     -6
  Partials          0          0

Powered by Codecov. Last updated by [d0e046c...7e50c86][cc-compare]
[cc-base-branch]: https://codecov.io/gh/travisbrown/iteratee/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/travisbrown/iteratee/compare/d0e046c4666fc7ad850a3f4cf0f00dff2f540b9d...7e50c8619f4b5448755cee401998257ed242efea
[cc-pull]: https://codecov.io/gh/travisbrown/iteratee/pull/87?src=pr


Add CI/CD pipeline with code coverage, linting, and code formatting

This pull request includes several updates to improve the project's configuration, documentation, and code quality. The most important changes include adding new dependencies and checks to the GitHub Actions workflow, updating the .Rbuildignore and .gitignore files, and enhancing the documentation.
GitHub Actions Workflow Improvements:

.github/workflows/R-CMD-check.yaml: Added new dependencies (covr, lintr, styler) and steps for running test coverage, linting, and code formatting checks.

Configuration Updates:

.Rbuildignore: Added .github and LICENSE.md to the ignore list.
.github/.gitignore: Added a rule to ignore HTML files.

Documentation Enhancements:

man/biodiversityhorizons-package.Rd: Added package documentation generated by roxygen2.
man/pipe.Rd: Added documentation for the pipe operator generated by roxygen2.

README Badge Addition:

README.Rmd: Added a badge for the R-CMD-check GitHub Actions workflow.Setup CI/CD pipeline using usethis  with code coverage, linting, and code formatting code.

Addressed https://github.com/uw-ssec/biodiversity-horizons/issues/3

approving but please address the comments