[SPARK-43365] Refactor Dockerfile and workflow based on base image

What changes were proposed in this pull request?
This PR changes Dockerfile and workflow based on base image to save space by sharing layers by having one image from another.
After this PR:

The spark / PySpark / SparkR related files extract into base image
Install PySpark / SparkR deps in PySpark / SparkR images.
Add the base image build step
Apply changes to template: ./add-dockerfiles.sh 3.4.0 to make it work.
This PR didn't contain changes on 3.3.X Dockerfiles to make PR more clear, the 3.3.x changes will be a separate PR when we address all comments for 3.4.0.

[1] https://github.com/docker-library/official-images/pull/13089?notification_referrer_id=NT_kwDOABp-orI0MzIwMzMwNzY5OjE3MzYzNTQ#issuecomment-1533540388
Why are the changes needed?
Address DOI comments, and also to save space by sharing layers by having one image from another.
Does this PR introduce any user-facing change?
No
How was this patch tested?
CI passed.

cc @HyukjinKwon @zhengruifeng

@HyukjinKwon Thanks! Merged.

Late LGTM


Implement AAAA records

I implement AAAA records in a simple way, but it works like charm. I also adjust the bash script for building and running the container to use the source files instead of the git repository and remove double files.

thanks for the contribution @GM-Alex, you beat me to it :)
it's going to take a few days for me to find the time to merge and test.  If you had an opportunity to rebase and resubmit, it would be much appreciated.
Can I add you to the CONTRIBUTORS.txt file?

I don't get the rebase thing, did I miss some update and worked on an outdated version?
Sure can you add me to the contributor. ;)

Hey Alex, there we a pull request which came in before yours which I merged, which means that now your changes won't merge without some manual intervention.
If you did happen to have an opportunity to pull and refactor I might suggest you split the work into multiple pulls to separate the concerns (move Dockerfile vs AAAA)
Thanks again for the good work.  I expect to find some time in the coming days.

Rebase done ;)


Feature/Minor changes

Just a few minor changes.
PR is drafted as it is still being worked on, but that way, you can track what has been changed and whatnot.

Thank you Eris for this PR of minor changes (+545 -501 lines)


[AutoPR machinelearningservices/resource-manager] [READY FOR DESIGN REVIEW] -> MachineLearningServices: Adding new API version and definition for AmlInstance compute type

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/7323

(message created by the CI based on PR content)


Request for additions to all.json

Site

https://www.simplcswap.io

clicking this ad redirects to simplcswap.io. Looks identical to simpleswap.io

simpleswąp.com

Looks identical to simpleswap.io

Will need to be checked, not sure if these are still valid.

Both sites seem to be taken down, I'm going to close this issue.


Bundlers

Try https://esbuild.github.io/ or https://parceljs.org/ both are very easy. It's generally much better to use a bundler anyways because (depending on your config) you'll get dead code elimination (tree shaking), avoid collisions, and you'll probably want to transpire you modern JS into something more supported anyways etc...
Hope that's helpful.
Here's a great article explaining build tools and why to use them and what the do. Helped me learn a lot about the why's and how they work: https://www.swyx.io/writing/jobs-of-js-build-tools/
Also didn't read over your codebase but you could try importing your JS scripts in your html with
<script type=module src=index.js />

Thanks @toreylittlefield . Will look into it soon.

Will check this after the first release


Beta-37 Agent Git Submodule Issue

We recently upgraded to the v3.0-beta.37 agent and have started getting this:
Warning: Failed to enumerate git submodules: Error running `/usr/bin/git submodule foreach --recursive git remote get-url origin`: exit status 1

It does not happen on a clean build, but does always happen when building a repo already checked out.

Any idea what version of git you have? The git remote get-url origin turns out to have been added more recently than I thought.

See https://github.com/buildkite/agent/issues/602

Sorry I missed that.

No problems, I'm sorry you ran into this!  In the meantime if you are able, a git upgrade will also fix it.


UMA2 U3D crash when switching for android platform

1.UMA2 U3D crash when switching for android platform
2.In the android platform, while importing UMA2 U3D no response

What does this mean? Do you get an error message?

I've managed to duplicate this, but it is not a UMA issue. The issue is that the textures are quite large, and Unity has to convert them, and this causes Unity to appear to lockup while the conversion is in process.  If you wait long enough, Unity will convert the textures, and start responding again. (took about 20 minutes for me).


Complete Developer Guide Appendix B: Use Cases

Complete use cases for the following user stories:

Adding a task
Adding a recurring task
Editing a task
Viewing completed tasks
Deleting a task
Undoing a previous action
Redoing a previous action
Finding a task
Changing a data storage location

Completed.


Replaced extended characters

There were a few rsquo characters that had been turned into question marks. I replaced them with a standard apostrophe instead.

Hi @JoBrad, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

JoBrad,
Thank you for updating the VBA content. Your contribution helps maintain the quality of our documentation.
Office Scale Team


Combining require.ensure with a dynamic require

I've finally found one possible solution to configuring how chunks should be created for my app, it looks something like this:
resolvers.index = function(name, cb) {
  require.ensure([
    "app/modules/foo/routes/a",
    "app/modules/foo/routes/b",
    "app/modules/foo/routes/c"
  ], function (require) {
    cb(require("app/modules/foo/routes/" + name));
  }, "index");
};

resolvers.review = function(name, cb) {
  require.ensure([
    "app/modules/foo/routes/d",
    "app/modules/foo/routes/e"
  ], function (require) {
    cb(require("app/modules/foo/routes/" + name));
  }, "review");
};

Here, the router will call resolvers.index("a"), resolvers.index("b") or resolvers.review("e"), etc. whenever it wants to load that page. require.ensure makes sure a named chunk is created for each group of routes. This is exactly what I want.
However, in the callback I need to require a specific route by name. Unfortunately require("app/modules/foo/routes/" + name) creates a dynamic context and forces everything into that chunk. So instead of having chunks "index" and "review", I get a chunk "index" with all of the routes in it. Any suggestions?

Webpack build should be have something like "build in memory/var script" to run extra requires.


[Request for External] Table 11400 "CBG Statement" - request-for-external

In table 11400 there are a lot of functions scoped as OnPrem while they are in my opinion not OnPrem specific.
We would like to request for external for at least the following 2 functions:
CLAccountNo()
GetName()
[Scope('OnPrem')]
procedure CLAccountNo() CLAccNo: Text[80]
var
    GLAccount: Record "G/L Account";
    BankAccount: Record "Bank Account";
    BankAccPostingGrp: Record "Bank Account Posting Group";
begin
    case "Account Type" of
        "Account Type"::"G/L Account":
            CLAccNo := "Account No.";
        "Account Type"::"Bank Account":
            case false of
                BankAccount.Get("Account No."):
                    CLAccNo := Text1000001;
                BankAccPostingGrp.Get(BankAccount."Bank Acc. Posting Group"):
                    CLAccNo := Text1000002;
                GLAccount.Get(BankAccPostingGrp."G/L Account No."):
                    CLAccNo := Text1000003
                else
                    CLAccNo := GLAccount."No.";
            end;
    end;
end;

[Scope('OnPrem')]
procedure GetName() Name: Text[100]
var
    GLAccount: Record "G/L Account";
    BankAccount: Record "Bank Account";
begin
    case "Account Type" of
        "Account Type"::"G/L Account":
            case false of
                GLAccount.Get("Account No."):
                    Name := Text1000004;
                else
                    Name := GLAccount.Name;
            end;
        "Account Type"::"Bank Account":
            case false of
                BankAccount.Get("Account No."):
                    Name := Text1000004;
                else
                    Name := BankAccount.Name;
            end;
    end;
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


[HTTP]:500 -  [CorrelationId]:ce55829e-50e8-6000-366c-8ca50e594c4f [Version]:<IP_ADDRESS>25 - The requested operation is part of an experimental feature that is not supported in the current environment.

Unable to access link : https://mylinkapps-admin.sharepoint.com/_layouts/15/online/AdminHome.aspx#/WebApiPermissionManagement

This most likely means that your tenant is not in the targeted release mode.
You can setup your tenant to be in the targeted release by following this link
Once you change to targeted release, you need to wait for a few hours to receive the updates and after that it will start working.

@gautamdsheth  Thanks.
These settings are already in place, but still it is not working
Are there any prerequisites for this.

Did you set target release to all users ?

yes..this is already in place

There is something wrong wit this page

This is indeed the tenant level targeted release setting. Notice that it takes up to 24 hours for that take into effect, so you will need to be patient after changing the setting. It also needs to be the full tenant setting, not just in individual persons.
If that is not working, we would need to get more details on the issue. This means that you should go to the page again, copy the exception with the Correlation ID. Share then timing, tenant name, and correlation ID with us, so that we can have a closer look at what goes wrong. Thanks for the additional details advance.

ok, I will wait and if not working I will provide the details.
@VesaJuvonen :-)

Yes, it works now (tenant settimg needs to be full tenant and not individual persons)
Thanks @VesaJuvonen


build: add scripts before release

Codecov Report

Merging #133 (336413f) into master (19386d3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #133   +/-   ##
=======================================
  Coverage   96.73%   96.73%
=======================================
  Files          23       23
  Lines         184      184
  Branches       36       36
=======================================
  Hits          178      178
  Misses          5        5
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19386d3...336413f. Read the comment docs.


[draft blog/idea] Why I Wish I Had a Control Plane for My Renovation

Which of these best describes you?

[ ] dbt Community member
[ ] Partner
[X] dbt Labs employee
[ ] Other

What's your idea for new content?
Why I Wish I Had a Control Plane for My Renovation
When I renovated my home, I chose to take on the role of owner-builder. It was a bold decision, but I wanted control over every aspect of the project. What I didn’t realise was just how complex and exhausting managing so many moving parts would be.
I had to coordinate multiple elements:

The architects, who designed the layout, interior, and exterior.
The architectural plans, which outlined what the house should look like.
The builders, who executed those plans.
The inspectors, councils, and energy raters, who checked whether everything met the required standards.

Each piece was critical—without the plans, there’s no shared vision; without the builders, the plans don’t come to life; and without inspections, mistakes go unnoticed. But as an inexperienced project manager, I was also the one responsible for stitching everything together. Architects handed me detailed plans, builders asked for clarifications, and inspectors flagged issues that were often too late to fix without extra costs or delays. What should have been quick, collaborative conversations turned into drawn-out processes because there was no unified system to keep everyone on the same page.
In many ways, this mirrors how data pipelines operate:

The architects are the engineers—designing how the pieces fit together.
The architectural plans are your dbt code—the models, tests, and configurations that define what your data should look like.
The builders are the compute layers (e.g., Snowflake, BigQuery, or Databricks) that execute those transformations.
The inspectors are the monitoring tools, which focus on retrospective insights like logs, job performance, and error rates.

Here’s the challenge: monitoring tools, by their nature, look backward. They’re great at telling you what happened, but they don’t help you plan or declare what should happen. And when these roles—plans, execution, and monitoring—are siloed, teams are left trying to manually stitch them together, often wasting time troubleshooting issues or coordinating workflows.

What Makes dbt Cloud Different
dbt Cloud unifies these perspectives into a single control plane, bridging proactive and retrospective capabilities:

Proactive planning: In dbt, you declare the desired state of your data before jobs even run—your architectural plans are baked into the pipeline.
Retrospective insights: dbt Cloud surfaces job logs, performance metrics, and test results, providing the same level of insight as traditional monitoring tools.

But the real power lies in how dbt integrates these two perspectives. Transformation logic (the plans) and monitoring (the inspections) are tightly connected, creating a continuous feedback loop where issues can be identified and resolved faster, and pipelines can be optimised more effectively.

Why Does This Matter?

The Silo Problem: Many organisations rely on separate tools for transformation and monitoring. This fragmentation creates blind spots, making it harder to identify and resolve issues.
Integrated Workflows: dbt Cloud eliminates these silos by connecting transformation and monitoring logic in one place. It doesn’t just report on what happened; it ties those insights directly to the proactive plans that define your pipeline.
Operational Confidence: With dbt Cloud, you can trust that your data pipelines are not only functional but aligned with your business goals, monitored in real-time, and easy to troubleshoot.

Why I Wish I Had a Control Plane for My Renovation
When I think back to my renovation, I realise how much smoother it would have been if I’d had a control plane for the entire process. There are firms that specialise in design-and-build projects, with in-house architects, engineers, and contractors. The beauty of these firms is that everything is under one roof, so you know they’re communicating seamlessly.
In my case, though, my architect, builder, and engineer were all completely separate, which meant I was the intermediary. I was the pigeon service shuttling information between them, and it was exhausting. Discussions that should have taken minutes stretched into weeks or even months because there was no centralised communication.
dbt Cloud is like having that design-and-build firm for your data pipelines. It’s the control plane that unites proactive planning with retrospective monitoring, eliminating silos and inefficiencies.
Where would you recommend this content live on the docs.getdbt.com?
Devhub blog

I'm a slow processor - so phrases like "dbt Cloud as the Control plane for your Data" takes a while to sink in. Lately I've been chewing on that phrase a bit more. I had an embryonic thought that dbt's approach to a control plan was incredibly unique vs what is out there. Specifically in regards of having both proactive and retrospective metadata developed and monitored in one place.
I thought this piece might be a helpful and relatable way to explain dbt as a Control plane
Totally open to feedback (and vito)


python3Packages.yeelight: 0.7.7 -> 0.7.8

Motivation for this change
Update to latest upstream release 0.7.8
Change log: https://gitlab.com/stavros/python-yeelight/-/blob/master/CHANGELOG.md
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 141986 run on x86_64-linux 1

3 packages built:

home-assistant
python38Packages.yeelight
python39Packages.yeelight


[BUG] Failing ERRO Failed to query DNS from host: lookup api.crc.testing on <IP_ADDRESS>:53: no such host

General information

OS: Linux
Hypervisor: KVM
Did you run crc setup before starting it (Yes/No)? YES

CRC version
crc version: 1.4.0+d5bb3a3
OpenShift version: 4.2.13 (embedded in binary)

CRC status
ERRO error: stat /home/pantinor/.crc/machines/crc/kubeconfig: no such file or directory
 - exit status 1

CRC config
[EMPTY]

Host Operating System
NAME=Fedora
VERSION="31 (Workstation Edition)"
ID=fedora
VERSION_ID=31
VERSION_CODENAME=""
PLATFORM_ID="platform:f31"
PRETTY_NAME="Fedora 31 (Workstation Edition)"
ANSI_COLOR="0;34"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:31"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f31/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=31
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=31
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation

Steps to reproduce

./crc start --log-level debug

Expected
Actual
Logs
https://gist.github.com/paoloantinori/7aa43fe441a8ff2ca2bb4605c981d897

Experiencing same behavior on CentOS Linux release 7.7.1908 (Core)

Please provide crc setup --log-level debug output, crc config view output, and any special networking configuration you might have done.

Hi @cfergeau, thanks for the reply!
As requested:
crc setup --log-level debug output
crc config view  ouput is empty
crc start --log-level debug output
I am not aware of any special networking configuration.
Thanks !

@johnpoth as per start logs I can see ERRO Failed to query DNS from host: lookup api.crc.testing on <IP_ADDRESS>:53: no such host  which means it might be issue that your dnsmasq is not running with networkManager. what is the outout of following ?
$ ls /etc/NetworkManager/conf.d
$ ls /etc/NetworkManager/dnsmasq.d/
$ cat /etc/resolv.conf

Hi @praveenkumar, thanks for your reply!
As requested:
ls /etc/NetworkManager/conf.d
crc-libvirt-dnsmasq.conf  crc-nm-dnsmasq.conf
ls /etc/NetworkManager/dnsmasq.d/
crc.conf
cat /etc/resolv.conf
# Generated by NetworkManager
search redhat.com cdg.redhat.com win.redhat.com
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
Thanks !

@johnpoth can you remove crc-libvirt-dnsmasq.conf from following directory /crc-libvirt-dnsmasq.conf looks like you have and old config.
@johnpoth also as per your logs, looks like the NM manager didn't start the dnsmasq and hence you are seeing that issue.
cat /etc/resolv.conf
# Generated by NetworkManager
search redhat.com cdg.redhat.com win.redhat.com
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>

which should be something like
$ cat /etc/resolv.conf
# Generated by NetworkManager
search redhat.com
nameserver <IP_ADDRESS>

Also can you provide us following commad the logs in the gist?
$ journalctl -b 0 -u NetworkManager

@praveenkumar thanks for the reply! I've deleted /crc-libvirt-dnsmasq.conf/crc-libvirt-dnsmasq.conf
journalctl -b 0 -u NetworkManager output

NetworkManager starts dnsmasq, but:
Feb 13 14:52:48 localhost.localdomain NetworkManager[2442]: <info>  [1581601968.0475] dns-plugin[0x5570865b7850]: starting dnsmasq...
Feb 13 14:52:48 localhost.localdomain dnsmasq[5026]: listening on lo(#1): <IP_ADDRESS>
Feb 13 14:52:48 localhost.localdomain NetworkManager[2442]: <warn>  [1581601968.0597] dns-mgr: could not commit DNS changes: Failed to rename file “/etc/resolv.conf.PBVXF0” to “/etc/resolv.conf”: g_rename() failed: Permission denied

Is there anything special with the permissions of your resolv.conf? Maybe some selinux issue?

Hi @cfergeau @praveenkumar, after playing with my network settings, it seems crc is working! Thanks a lot!

Hi, I'm back at this. I've revied my network config but couldn't find anything strange.
See most recent logs:
https://gist.github.com/paoloantinori/65205b531b4d0f9380acce27fcdf6aa0
11:49:17 (..-linux-1.6.0-amd64)$ cat /etc/resolv.conf
# Generated by NetworkManager
search station
nameserver <IP_ADDRESS>

11:51:43 (..-linux-1.6.0-amd64)$ systemctl status dnsmasq.service
● dnsmasq.service - DNS caching server.
   Loaded: loaded (/usr/lib/systemd/system/dnsmasq.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

11:52:07 (..-linux-1.6.0-amd64)$ ls /etc/NetworkManager/dnsmasq.d/
crc.conf

11:52:28 (..-linux-1.6.0-amd64)$ cat /etc/NetworkManager/dnsmasq.d/crc.conf
server=/apps-crc.testing/<IP_ADDRESS>
server=/crc.testing/<IP_ADDRESS>

11:52:47 (..-linux-1.6.0-amd64)$ ps aux | ag dnsmas
dnsmasq     2640  0.0  0.0  22936  2272 ?        S    11:09   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
root        2641  0.0  0.0  22832    40 ?        S    11:09   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
dnsmasq     2685  0.0  0.0  22860  1940 ?        S    11:09   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/docker-machines.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
root        2686  0.0  0.0  22832   196 ?        S    11:09   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/docker-machines.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
dnsmasq     2732  0.0  0.0  22860  1992 ?        S    11:09   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/crc.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
root        2733  0.0  0.0  22832  1632 ?        S    11:09   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/crc.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper

Any clue on how to debug this further?

Network Manager's dnsmasq instance does not seem to be started, can you enable level=TRACE in /etc/NetworkManager/NetworkManager.conf, restart NetworkManager with systemctl, and provide journalctl -b 0 -u NetworkManager ?

Thanks for the help.
here you go: https://gist.github.com/paoloantinori/9c1ef02e9490e78dc496846c8455a618

Feb 19 15:01:24 trane NetworkManager[21058]: dnsmasq: failed to create listening socket for <IP_ADDRESS>: Address already in use

This seems to be why NM's dnsmasq instance is not running, do you know what is listening on <IP_ADDRESS>:53?

15:59:18 (..-linux-1.6.0-amd64)$ sudo netstat -putan | ag <IP_ADDRESS>:53
tcp        0      0 <IP_ADDRESS>:53            <IP_ADDRESS>:*               LISTEN      2452/unbound

15:59:44 (..-linux-1.6.0-amd64)$ ps aux | ag 2452
unbound     2452  0.0  0.0<PHONE_NUMBER>6 ?        Ssl  11:09   0:01 /usr/sbin/unbound -d

no clue what is supposed to do

That's a DNS server, but I don't know why it's installed/running on your machine
https://nlnetlabs.nl/projects/unbound/about/

Thank you, that was it.
Thank you everyone involved with this remote debugging activity! Much appreciated! 🙏
I don't think there is a way to generalize, but maybe a useful FAQ could be: verify you don't have any other process listening on <IP_ADDRESS>:53


datepublisher_response_processor throws exception when publication_end_date is timezone aware

If a page has a timezone-aware publication_end_date the datepublisher_response_processor will try to subtract datetime.now() from it and throw TypeError: can't subtract offset-naive and offset-aware datetimes, leading to a server error rather than  the desired page. Changing to timezone.now() fixes it for me, but I haven't tested it in other situations.

Not sure whether this is fixed? It probably is...
Please reopen if not. Thanks!

I think this is fixed apart from the WARNING/TODO here -- https://github.com/DrMeers/feincms/blob/1707da9e884d1adbfc6911c71a1969abab8e1d89/feincms/module/extensions/datepublisher.py#L54
-- only occurs if the make_aware conversion results in a time which falls in the missing/extra hour during the night when daylight savings changes...


Add global launch and configuration files

Description

Depends on the following PRs:

[ ] https://github.com/UBCSailbot/boat_simulator/pull/28
[ ] https://github.com/UBCSailbot/network_systems/pull/31

If not merged, use the following branches:

local_pathfinding: main
boat_simulator: user/dfriend01/17-overhaul-launch-files
network_systems: user/dfriend01/update_launch_file_name

This PR aims to streamline the process of running all packages while having a single source for global configuration and launching.
To Dos

[ ] Move launch file to somewhere insides src/, as this is where the linters run
[ ] Move global config file from local_pathfinding to this repository
[ ] Read global config file in main_launch.py

Verification

[ ] Global launch file launches all packages as expected and all ROS topics show up in ros2 topic list.

Resources

Invoking children launch files from a parent launch file
Using launch arguments directly in parent launch file

Adding arguments
To do this, I believe that the steps are the following:

Edit the function get_include_launch_descriptions() in main_launch.py to accept launch arguments and include them in the launch include a description. I believe section 2 of this link shows you what that looks like.
In each of the children launch files, make sure that the global parameters are declared with the DeclareLaunchArgument class and include them in the LaunchDescription (like what local pathfinding does right now).
If you want to add any more global arguments, follow the instructions on line 17 of main_launch.py and repeat step 2.

@patrick-5546 It was decided that the following will be done:

Global configuration files and package-specific configuration files will be stored in sailbot workspace
Both the main launch file and the children will accept two arguments

The path to the global config file to be used
The path to the package-specific config file to be used

If launched via the main launch file, whatever the main launch file specifies will override the arguments in the children launch files.

After this is merged let's create a minor release


Flutter web

How can we use the flutter web?

Flutter web is supported out of the box. Just make sure you don't configure any HttpClientAdapter or  use BrowserHttpClientAdapter.
If you have a question with a little more information, or a failing example, the don't hesitate to reopen the issue.


Update Takt api to be more simple and work correctly on 8.1

Hello!
Proposed changes fix behaviors described in #11 and readme inaccuracies described in #12
When I first tried to use Takt, I stumbled upon few behaviors that weren't  described in readme, and decided to fix them by making api more understandable and easier to use.

@aleien
Thanks.


#2156 - added support for glyphs

Hey, thanks for implementing this and sorry I didn't come to this sooner.
I wrote up some of my thoughts about escape characters in #2354. I think we should wait until at least the first step is implemented, before we add proper uses for escape characters.
Also, escaping should happen in the config parser and not when labels are loaded.

Hey, thanks for implementing this and sorry I didn't come to this sooner.
I wrote up some of my thoughts about escape characters in #2354. I think we should wait until at least the first step is implemented, before we add proper uses for escape characters.
Also, escaping should happen in the config parser and not when labels are loaded.


Warn when trying to navigate/leave with unsaved changes

This is  a feature request.

Prerequisites

[x] Are you running the latest version?
[x] Are you reporting to the correct repository?
[x] Did you check the documentation?
[x] Did you perform a cursory search?

Description

Alert when trying to navigate away from the form with unsaved changes
Steps to Reproduce

Edit any form
Navigate away without clicking save

Expected behavior

Warning before leaving/closing the page losing changes
Note: 2-sided: navigate within vue-router & navigating externally (i.e. page refresh and/or close)
Actual behavior

Navigating away, all changes are lost

I really don't know how we should approach this, @gandesc any ideas?
Something I considered but not concluded in the past was to keep-alive the forms in some scenarios, or use the localStorage to keep info for dirty forms
Meanwhile I guess we could display a flag when a form is dirty ...

I was just thinking about a good approach to implement this.
I think a decent approach would be to have a modified flag of sorts on each input field; then have VueForm keep track of these flags; if "more than 0 occurences"; calling back to onbeforeunload for browser side & using Navigation guards for vue-side requests
// Enable navigation prompt
window.onbeforeunload = function() {
    return true;
};
// Remove navigation prompt
window.onbeforeunload = null;```

I've built an initial implementation of this; I'll just share the changes in this comment as it's most likely not PR worthy yet. Will provide a PR after feedback though:
In FormActions.vue after successfully submitting the form, add:
                    store.commit('setPreventUnload',  false);

Create a new before middleware (names preventunload.js in the examples) in resources/js/middleware/before:
import store from "../../store";

export default (to, from, next) => {
    console.log('beforeEach', store.state.preventUnload);
    if(store.state.preventUnload) {
        console.log('preventUnload');
        const answer = window.confirm('Do you really want to leave? you have unsaved changes!');
        if (answer) {
            store.commit('setPreventUnload',  false);
            next()
        } else {
            console.log('preventing next');
            return next(false)
        }
    } else next();
};

In router.js add this file as a beforeEach hook, next to the pre-existing beforeEach:
router.beforeEach(preventunload);

(note: I added it into the before middleware first but that seemed to ignore next(false) + the documented way is actually to add multiple hooks in this manner)
In store.js, add...
the following state:
        preventUnload: false,

the following mutation to mutate above state
        setPreventUnload: (state, value) => (state.preventUnload = value),

and (in my case) at the end of the initialise action:
            window.onbeforeunload = () => {
                // string not shown in modern browsers; but just a boolean doesn't work
                return (this.preventUnload) ? 'Unsaved changes' : false;
            }

In VueForm.vue add a method and hook it to the update event of the <form-field:
                        @update="changesMade">

...
        changesMade() {
            this.$store.commit('setPreventUnload',  true);
        }

To pass along this event from the fieldType components, re-emit the update event in FormField.vue:
            @update="$emit('update')"

Finally, as I was testing with TextareaFields (would be required for all field types in final implementation) add a keydown event to TextareaField.vue's textarea:
            @keydown="$emit('update');"/>

Making above changes causes any (user)change to a formfield (tested with textarea, others should behave the same) causes the store flag to be set; showing a dialog either through Vue (window.confirm) or the browser (onbeforeunload) to (respectively) prevent in-vue or browser-side navigation
sidenote: this comment is bigger than I expected it to be 😅

VueForm.vue is meant to be used outside Enso too. Your approach breaks this.
I don't think we need the store nor the global middleware to get this working.
We should:

add to every create / edit route component and a preventLeave boolean flag
add to every create / edit route component a beforeLeave navigation guard from which we have access to this.preventLeave, where we will have the logic for stopping the navigation
build a computed boolean isDirty on the vueForm, use it in the future for other things
every time isDirty changes emit a changed event with the boolean value of isDirty
listen on the event from the Edit.vue / Create.vue pages and manipulate the preventLeave flag
handle window.onbeforeunload from the VueForm also.

The changes on Edit.vue would be by default present in StructureManager's stub too.
I hope I didn't miss anything.

I don't think we need the store nor the global middleware to get this working.

To do this DRY we do, as Vue router (logically) doesn't handle beforeRouteLeave of subcomponents & Vue itself doesn't have a mechanism to prevent this either.
This would mean (as you indicated) that every route would need the exact same code to handle this, which is completely against modern coding principles.

VueForm.vue is meant to be used outside Enso too. Your approach breaks this.

Hence also just a PoC; we could pass the prop to it's parent to affect the store/trigger there instead; still allowing full use outside Enso without real assumptions (the dialogs themselves can't be implemented without breaking encapsulation anyway; so that's up to the user)

I'm used already to your style of responding so I'll skip the obvious.
I didn't say every route but every route component (eg Edit.vue & Create.vue)
The route guard would look like:
beforeRouteEnter (to, from, next) {
    this.preventLeave();
},

beforeRouteEnter (to, from, next) {
   preventLeave(this.preventLeave);
},

The actual logic for (this.)beforeLeave() would be in only one place.

I didn't say every route but every route component (eg Edit.vue & Create.vue)

Sorry, that's what I meant 😅

The actual logic for (this.)beforeLeave() would be in only one place.

Agreed, but as most routes are likely to have something that has to be saved it just makes more sense to implement this globally to reduce clutter. But that's just my opinion though.

We have to decide if this is something vueform specific or it’s a global feature of the framework.
If it’s form related (form pages) we should not need to control from the local component a global mechanism that affects ONLY that specific local component.

I suppose it lies somewhere in between...
I think the handling is rather something global; and although the above implementation only affects vueform, any other form-like element that sets the preventUnload will also prevent unloading
example global use case: non-VueForm based flow that is usually expected to be completed before navigating away, could re-use the state & globally defined unload handlers
sidenote: while doing more testing I discovered the current onbeforeunload (window, not vue-router) method used above didn't always work. instead I've modified the mutation to ALSO set window.preventUnload and modified the window.onbeforeunload = ... to:

            window.addEventListener('beforeunload', function (e) {
                if(window.preventUnload) {
                    e.preventDefault();
                    e.returnValue = '';
                }
            });

(I still believe it should be kept in state/using a mutation so you can properly see changes in debugging tools)

I will add the feature at the framework level (globally) with a store flag and two mutators (prevent..., allow...)
We will have a middleware for the router and beforeunload will be managed from the Default.vue layout component.
VueForm.vue will have a computed isDirty prop and will emit two events, changed and unchanged.
I'm thinking to include the form fetching request in the VueForm component, like VueTable. Probably will rename VueFormSS into something else and use it only as a wrapper for Enso that will only handle localisation, determining the route/path and preventing navigation.

@aocneanu Can I help to get this implemented, or are you working on this?
We have a couple forms coming up with serious risk of forgetting to save due to the inherent complexity involved, and I want to avoid saving the model on every action.

@jlsjonas Please do, we should start discussing about the best approach for adding this.
In conjunction with the bookmarks this could bring a lot of value to Enso.

@aocneanu commented on Nov 30, 2018, 11:12 AM GMT+1:

I will add the feature at the framework level (globally) with a store flag and two mutators (prevent..., allow...)
We will have a middleware for the router and beforeunload will be managed from the Default.vue layout component.
VueForm.vue will have a computed isDirty prop and will emit two events, changed and unchanged.
I'm thinking to include the form fetching request in the VueForm component, like VueTable. Probably will rename VueFormSS into something else and use it only as a wrapper for Enso that will only handle localisation, determining the route/path and preventing navigation.

That still looks like a valid strategy to me, VueFormSS was reworked to EnsoForm where all we require should be in VueForm already.
For Router-based prevention we could use a pretty dialog via import { Dialog } from 'buefy/dist/components/dialog'; (using that one in production alongside Enso already) & the browser-based prevention is basically just a flag with an internal dialog we have no control over anymore (due to the related abuse)

addressed in the latest version of forms


[vpc] update dynamic-subnet module version

what
Update dynamic-subnets module version to fix issue https://github.com/cloudposse/terraform-aws-components/issues/1047
why
Version 2.3.0 of dynamic-subnets does not support the new opt-in Melbourne region.
references
This change in utils module adds support for Melbourne (amongst other regions) - https://github.com/cloudposse/terraform-aws-utils/pull/26
And this is the version of dynamic-subnets where it was updated to use the updated utils module.  https://github.com/cloudposse/terraform-aws-dynamic-subnets/releases/tag/2.4.0
This PR closes #1047

@mtb-xt Please rebase branch.

@mtb-xt Please rebase branch.

It is done

@goruha rebased again

This branch is out-of-date with the base branch

I'm not sure why we are seeing this. This setting is disabled, yet GH is still enforcing it here.


Connect database problem

Bug description
Hi, all.
I try to deploy the defectDojo into production and  I changed all password.
Into docker logs I get:
initializer_1   |   File "/usr/local/lib/python3.7/subprocess.py", line 363, in check_call initializer_1   |     raise CalledProcessError(retcode, cmd) initializer_1   | subprocess.CalledProcessError: Command '['mysql', '--user=defectdojo', '--password=defectdojo', '--host=mysql', '--port=3306', 'defectdojo']' returned non-zero exit status 1.
Steps to reproduce
Steps to reproduce the behavior:

Go to docker-compose.yml
Change MYSQL_PASSWORD: ${DD_DATABASE_PASSWORD:-defectdojo} variable. Setup any password, for exmpl:
MYSQL_PASSWORD: ${DD_DATABASE_PASSWORD:-12345}
Also I tried to change ENV in settings.dist.py and Dockerfile.django
docker-compose up --build.
See error described into bug description.

Expected behavior
A clear and concise description of what you expected to happen.
Deployment method (select with an X)

[ ] Kubernetes

[x]  Docker

[ ] setup.bash / legacy-setup.bash

Environment information

DefectDojo Commit Message:

git show origin/master -s --format="[%ci] %h: %s [%d]"
[2020-06-16 23:28:05 +0200] f93e24ef: removing dev branch and add security section [ (origin/master, origin/HEAD)]

Screenshots (optional)

Is this with a clean database or did you change the password after some earlier build/run which created the database? Can you run docker system prune --volumes and try again? (This will remove ALL docker volumes, containers, etc)

@valentijnscholten Yes. I did docker system prune --volumes and have the same error.
Also. I went to mysql container and login to mysql as defectdojo user and password 1234, that was setup into docker-compose.yaml

I found trouble.
It's DD_DATABASE_URL: ${DD_DATABASE_URL:-mysql://defectdojo:defectdojo@mysql:3306/defectdojo} ENV variable.
I setuped it as DD_DATABASE_URL: ${DD_DATABASE_URL:-mysql://defectdojo:12345@mysql:3306/defectdojo} (for my example) and it working fine.
I think you'll should setup this variables in dependences from
MYSQL_USER, MYSQL_PASSWORD and MYSQL_DATABASE env variables.


S3 VPC Endpoint for public subnets

Hello there,
Can someone explain to me why S3 Endpoint is enabled for public subnets?
As far as I understand AWS, this is not necessary. Or at least, it should be an option.
Would it be possible to add an extra variable like enable_public_s3_endpoint ?
https://github.com/terraform-aws-modules/terraform-aws-vpc/blob/23b0a02d2a3225a05ddf5f31d9a32728bf5505c3/vpc-endpoints.tf#L32-L37

I can open a PR to the enable_public_s3_endpoint flag with default to true not to break the module but the main reason to a VPC Endpoint in a public subnet is probably compliance/security
you can restrict access to bucket based on vpc endpoint id


Staking & Calming Issue

There is a problem with Staking & Calming rewards, which needs to be fixed. After clicking on cancel staking, the tokens are not correctly returned to the account.

This is not a support channel, please head to https://discord.tea.xyz/

Hello @babulsahoo20  A support ticket has been opened for you. Kindly Talk to Support on the Live support page so that your issue can be looked into. CLICK HERE


Fira sans is broken in Chrome and Firefox on Linux

On Debian testing at least but probably on other distros as well. Could be also broken on Mac, the issue with Fira sans was revealed in habrahabr.ru redesign (worked fine on Windows but was totally invisible on Mac/Linux). It looks like this:

Disabling font-family in body reveals the text. Please replace the font with the working version.

It's getting it from fonts.googleapis.com, so I imagine it's a local issue.

Google isn't impeccable, and it's not local. Here's the issue: https://github.com/google/fonts/issues/10

Ha, thanks for shattering my worldview. Just switched to Open Sans - let me know if there's still any problems.

Thanks, now it looks just fine!


Update xtend version

This library relies on version 0.2.6 of xtend which itself relies on a deprecated version of object-keys.  This dependency should be updated (or removed if not needed).

👍


docs(Storage): removes ready() method

Users don't have to call the ready() method.

Hey @MTechDE thanks for this (and all your other) PRs.
The Storage doc is actually generated from source
https://github.com/driftyco/ionic-storage/blob/master/src/storage.ts#L21
So a change would need to be made there,

Ah okay! Thank you


Update feature_request.yml

:tada: This PR is included in version 0.2.0-dev.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 0.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Automated PR: Cookstyle Changes

Hey!
I ran Cookstyle 7.26.1 against this repo and here are the results.
This repo was selected due to the topics of chef-cookbook
Changes
Issues found and resolved with resources/agent.rb

1:1 refactor: Chef/Deprecations/ResourceWithoutUnifiedTrue - Set unified_mode true in Chef Infra Client 15.3+ custom resources to ensure they work correctly in Chef Infra Client 18 (April 2022) when Unified Mode becomes the default. (https://docs.chef.io/workstation/cookstyle/chef_deprecations_resourcewithoutunifiedtrue)

Issues found and resolved with resources/user.rb

1:1 refactor: Chef/Deprecations/ResourceWithoutUnifiedTrue - Set unified_mode true in Chef Infra Client 15.3+ custom resources to ensure they work correctly in Chef Infra Client 18 (April 2022) when Unified Mode becomes the default. (https://docs.chef.io/workstation/cookstyle/chef_deprecations_resourcewithoutunifiedtrue)

Before merging we need to validate the minimum chef version in the metadata.rb file

Released as: 5.0.0


fix log rotation

Log rotation was broken when we want to log on both the file and console.
==========
Testing Done
Tested enforcer with custom log line in a loop. The file in /var/log/enforcerd at 10MB was backed up with the timestamp and a new file was used to write logs.

/build


[Outlook] (preview) Update event-based activation walkthrough

Docs Build status updates of commit 466179b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/images/outlook-beta-registry-key.png
:white_check_mark:Succeeded
View

docs/outlook/autolaunch.md
:white_check_mark:Succeeded
View

docs/outlook/debug-autolaunch.md
:white_check_mark:Succeeded
View

docs/outlook/sideload-outlook-add-ins-for-testing.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 1d779c9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/images/outlook-beta-registry-key.png
:white_check_mark:Succeeded
View

docs/outlook/autolaunch.md
:white_check_mark:Succeeded
View

docs/outlook/debug-autolaunch.md
:white_check_mark:Succeeded
View

docs/outlook/sideload-outlook-add-ins-for-testing.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


webdriver is not yet working for chrome version 119.0.6045

I have a problem, that the library does not support chrome version 119.0.6045, please help to check my error.
[WDM] - ====== WebDriver manager ======
[WDM] - Current google-chrome version is 119.0.6045
[WDM] - Get LATEST chromedriver version for 119.0.6045 google-chrome
There is no such driver by url https://chromedriver.storage.googleapis.com/LATEST_RELEASE_119.0.6045
This is what I got from the terminal.

upgrade 'webdriver-manager==4.0.1',

I believe you will need python>3.8 to run the code? You could give it a try. I'm running on Python 3.11

2024-04-06 01:26:30,277 - facebook_page_scraper.driver_initialization - INFO - Using: IP:PORT
[WDM] - Current google-chrome version is 123.0.6312
[WDM] - Get LATEST driver version for 123.0.6312
Im struggling with the same problem
even though I already downloaded the driver, and updated the packages
!pip install selenium==4.1.0
!pip install webdriver-manager==4.0.1
!pip install selenium-wire==5.1.0
!pip install python-dateutil==2.8.2


Verify project failing

Command:
rancher-projects --rancher-server 'https://rancher.support.tools' --rancher-access-key 'token-xxxxx' --rancher-secret-key 'xxxxxxx' --cluster-name a1-ops-dev --project-name "SupportTools" --namespace supporttools-dev --create-kubeconfig true --kubeconfig "temp"

Output:
Verifying access to Rancher server...
Successfully authenticated to https://rancher.support.tools
Verifying cluster a1-ops-dev...
Successfully found cluster a1-ops-dev
Getting cluster ID for a1-ops-dev...
Cluster ID: c-m-ffg95w9z
Verifying project SupportTools...
2024/02/21 01:53:27 Error verifying project 'SupportTools': failed to parse project data for SupportTools: json: cannot unmarshal object into Go value of type []map[string]interface {}
2024/02/21 01:53:27 Error handling project 'SupportTools': error verifying project 'SupportTools': failed to parse project data for SupportTools: json: cannot unmarshal object into Go value of type []map[string]interface {}
2024/02/21 01:53:27 Failed to handle single cluster: error handling project 'SupportTools': error verifying project 'SupportTools': failed to parse project data for SupportTools: json: cannot unmarshal object into Go value of type []map[string]interface {}

Fixed under https://github.com/SupportTools/rancher-projects/commit/73cb674c52df3dd7d615db33fac65f21008e1a41


DRAFT: Update ci

Updating linters and adding python 3.12 to workflow to see if everything works.

3.12 seems to crash because of the removal of distutils.
https://docs.python.org/3.12/whatsnew/3.12.html#distutils

Seem to be due to numpy not supporting 3.12 until 1.26.0 which is currently excluded from our requirements.
https://numpy.org/devdocs/release/1.26.0-notes.html

I have updated the numpy dependency, maybe that now makes it work with 3.12.
I also added pymend to the pre-commit configuration. Left me know what you think.

Also i feel that our current setup with the installation is a bit messy. We have the requirements.txt, setup.py and pyproject.toml and all of them specify the dependencies.

Updated all dependencies to the newest versions on pypi. Lets see if this builds and if all tests still pass.

3.12 issue seem to now be shapely which does not have a wheel for 3.12 yet on pypi.

Totally changed everything for v2. If you have any suggestions, fire away either in a PR, comments or Discord, I trust your judgement :)


Possible bug with parse_interactive and unknown_param_0?

When calling .exhaust_lexer() I'm seeing differing behavior between Python and Javascript.
I think I've narrowed it down to this bit of code:
https://github.com/lark-parser/Lark.js/blob/master/larkjs/lark.js#L3761
Which sets LexerThread.state to:
{
unknown_param_0: text,
      start: start,
}

Later on, next_token is called here, which calls this:
while (line_ctr.char_pos < lex_state.text.length) {

Which seems to imply that it expects text to be a string, but in fact text is { unknown_param_0: text, start: start, }.

Conversely, what appears to be the relevant python code is expecting a string.

I tried modifying the relevant Javascript bit to be:
  parse_interactive(text = null, start = null) {
    return this.parser.parse_interactive(text, start);
  }

And it appears to work now. But I'm not familiar enough with the code to know whether doing so introduces other bugs.

It's just a bug in the automatic conversion. I think this fix should be okay, though I'll have to check to make sure.
You're welcome to open a PR for it.

PR opened here: https://github.com/lark-parser/Lark.js/pull/44


Add SSL support

Currently, there is no support for SSL so the SSL middleware needs to be commented out in the deployed buffalo app for it to work. I would like to add SSL support via docker with something like Caddy possibly with https://hub.docker.com/r/abiosoft/caddy/.
Not sure if I want to make this automatic or triggered by a flag.

Implemented here: https://github.com/wolves/buffalo-ocean/releases/tag/1.0.0-rc.3


Add section about the docker image

Closes #11 .

I have read the CLA Document and I hereby sign the CLA


parse motorroad=yes in CarFlagEncoder

This pull request detects motorroad=yes tag in highways and fetch the speed accordingly.

Sorry I am a bit busy this week, send you the CLA after I improve the pull
request first.
On Friday 02 December 2016 11:48:07 Peter wrote:

karussell commented on this pull request.

@@ -158,6 +158,9 @@ public int defineWayBits(int index, int shift) {

 protected double getSpeed(ReaderWay way) {
     String highwayValue = way.getTag("highway");

   if (way.hasTag("motorroad", "yes")) {

Furthermore would you mind to send me an email regarding [our
CLA](https://github.com/graphhopper/graphhopper/blob/master/.github/CONTRIB
UTING.md#license-agreement)?

--
Sent from KMail

CLA attached
On Friday 02 December 2016 11:48:07 Peter wrote:

karussell commented on this pull request.

@@ -158,6 +158,9 @@ public int defineWayBits(int index, int shift) {

 protected double getSpeed(ReaderWay way) {
     String highwayValue = way.getTag("highway");

   if (way.hasTag("motorroad", "yes")) {

Furthermore would you mind to send me an email regarding [our
CLA](https://github.com/graphhopper/graphhopper/blob/master/.github/CONTRIB
UTING.md#license-agreement)?

--
Sent from KMail

Can you please send it via email to us? It seems to be missing here.

@karussell have you received it yet?

As I said in the comment: there was no attachment neither here nor in the email notification. Would you mind to sent it to the linked email instead of github email?

On Monday 06 February 2017 08:27:50 Peter wrote:

As I said in the comment: there was no attachment neither here nor in the
email notification. Would you mind to sent it to the linked email instead
of github email?

I have sent it already to the graphhopper email on 12 December
Sent from KMail

@boldtrn is this good to merge?


Add ND support to EntropyBottleneck compress and decompress, add 0D support

Hello again :)
I am sending this pull request to complete the previous one.
Specifically, I added the following changes:

0D support for compress, decompress, forward
ND support for compress, decompress
Associated unit tests

All tests are passing on my machine.
Best,
Maurice

Awesome! Thanks Maurice. I'll merge this soon.

merged :-).


Include and Take in same query causing materialization issue

In below situation, attempting to load a many-to-many object with navigation and its associated relation. EF Version 7 RC1:
 var query = from applicationUsers in _appDbContext.ApplicationUser
                            .Include(a => a.UserSpecializations)  // <-- This is a mapping object
                            .ThenInclude(b => b.Specialization) // <-- this is the collection of specializations belonging to app user
                        where applicationUsers.UserSpecializations.Any(s => s.SpecializationID == Id)
                        select applicationUsers;

            // TODO can't figure out why .Take(10) is preventing eager load of Specialization ICollection in ApplicationUser parent object.  If I remove Take(10), I get eager load as specified by above .Include
            var results = query
                .AsNoTracking()
                .Take(10)
                .ToList();

Workaround:
var results = query
                .AsNoTracking()
                .Skip(0) // <-- took me several days to figure out if I simply add Skip, everything works..
                .Take(10)
                .ToList();

@azamk100 I haven't been able to reproduce this issue with either RC1 or RC2 bits and the information you provided. I put together some code that includes your queries executing against a simplistic model. Can you spot any significant difference that could cause the query to fail to eager load in your application and work with this code?
using System;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.Linq;
using Microsoft.Data.Entity; // change this to "using Microsoft.EntityFrameworkCore;" for RC2

namespace Repro5097
{
    class Program
    {
        static void Main(string[] args)
        {
            var enginnerId = 0;
            using (var context = new MyContext("deleteme"))
            {
                context.Database.EnsureDeleted();
                context.Database.EnsureCreated();
                var maria = new User { Name = "Maria" };
                var ana = new User { Name = "Ana" };
                var engineer = new Specialization { Name = "Engineer" };
                var scientist = new Specialization { Name = "Scientist" };
                var artist = new Specialization { Name = "Artist" };
                context.Add(maria);
                context.Add(ana);
                context.Add(engineer);
                context.Add(scientist);
                context.Add(artist);
                context.SaveChanges();
                context.Add(new UserSpecialization { UserId = maria.Id, SpecializationId = engineer.Id });
                context.Add(new UserSpecialization { UserId = maria.Id, SpecializationId = scientist.Id });
                context.Add(new UserSpecialization { UserId = ana.Id, SpecializationId = engineer.Id });
                context.Add(new UserSpecialization { UserId = ana.Id, SpecializationId = artist.Id });
                context.SaveChanges();
                enginnerId = engineer.Id;
            }

            using (var context = new MyContext("deleteme"))
            {
                var engineers = from users in context.Users
                            .Include(a => a.UserSpecializations)
                            .ThenInclude(b => b.Specialization)
                            where users.UserSpecializations.Any(s => s.SpecializationId == enginnerId)
                            select users;

                var results = engineers
                    .AsNoTracking()
                    //.Skip(0)
                    .Take(10)
                    .ToList();

                foreach (var user in results)
                {
                    Console.Write($"User {user.Name} loaded with specializations ");
                    foreach (var userSpecialization in user.UserSpecializations)
                    {
                        Console.Write($"{userSpecialization.Specialization.Name} ");
                    }
                    Console.WriteLine();
                }
            }
        }
    }

    public class MyContext : DbContext
    {
        private readonly string _connectionString;

        public MyContext(string database)
        {
            _connectionString =
                new SqlConnectionStringBuilder
                {
                    DataSource = @"(localdb)\mssqllocaldb",
                    InitialCatalog = database,
                    IntegratedSecurity = true
                }.ToString();
        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlServer(_connectionString);
        }
        public DbSet<User> Users { get; set; }
        public DbSet<Specialization> Specializations { get; set; }
        public DbSet<UserSpecialization> UserSpecializations { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<UserSpecialization>().HasKey(x => new { x.UserId, x.SpecializationId });
        }
    }
    public class User
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public ICollection<UserSpecialization> UserSpecializations { get; set; }
    }

    public class Specialization
    {
        public int Id { get; set; }
        public string Name { get; set; }
    }

    public class UserSpecialization
    {
        public int UserId { get; set; }
        public int SpecializationId { get; set; }
        public Specialization Specialization { get; set; }
    }
}

Clearing up for triage.

Diego, apologize I haven't had a chance to check your suggestion.
I'll have a reply by tomorrow.
On Thu, Apr 21, 2016 at 4:37 PM Diego Vega<EMAIL_ADDRESS>wrote:

Clearing up for triage.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/aspnet/EntityFramework/issues/5097#issuecomment-213103820

Closing as we think that it's been fixed, feel free to reply if you still see the issue and we will reopen.