RIDDL → Diagrams

A wide variety of diagrams could be produced from RIDDL source input. This needs to be done separately from documentation so it can be reused in contexts other than documentation.

Not tracking epics any more


Please add cohere-ai support the Chinesa language

What feature(s) would you like to see?
Cohere-ai support the Chinesa language.
Cohere-ai will have lots of Chinese user and it will bring convenience for non-english speakers.
Download and Add Chinese data for model to trainning.Add Foreign Language Translation function
Additional information
No response

Hi, thank you for the feedback currently, command-r-plus has some multilingual capabilities for Simplified Chinese, more details: https://docs.cohere.com/docs/command-r-plus#multilingual-capabilities


fix: pfe-card header and footer slot issues

Fixes #860
Testing Instructions

Navigate to the pfe-card demo page
Open the developer tools
Drop in the following code into the console and execute it

const card = document.createElement("pfe-card");
card.textContent = "Test Card";
document.body.appendChild(card);

Make sure the card is visible at the bottom of the page
Add the following code into the console and execute it

const header = document.createElement("h2");
header.setAttribute("slot", "pfe-card--header");
header.textContent = "Card Header";
card.prepend(header);

Validate that the new header shows up in the card.

Browser testing

Tested in Firefox, Safari, Chrome, Edge 80+, Edge 18, and IE11

I also did an npm link to a React application on my local computer and validated that the header and footer slots are now rendering properly.
Before the fix

After the fix

@kylebuch8 Dang these are such great updates and test cases but I've had a PR opened that refactors the card styles and this would be seriously hard to merge with it: https://github.com/patternfly/patternfly-elements/pull/592


Placeholders

This PR introduces a PlaceholdersHelper to render placeholders introduced in bedita/bedita#1776.

Closing in favour of https://github.com/chialab/bedita-frontend-kit/pull/27.
We will keep this branch until projects are not fully migrated.


Support finer controls for row visibility in the GUI

Description
We are going to support double height cells (#3099 ) which means we will have double height rows as well as single height rows overlapping each other.  The current GUI control

isn't fine grained enough.
For example:

Suggested Solution
We should have the ability to turn on/off rows by SITE.  The rows item could become a twist-open like Layers.
Additional Context
@gadfort do you have time for this?

@maliberty yeah, when I was working on the pad ring thing, that would have been helpful as well.
Any thoughts about color? keep them the same for all rows?


RuntimeError: ["Command failed. No directory or zip file found: '/app/application'", '']

When running documentation example from how-to https://pyvespa.readthedocs.io/en/latest/howto/deploy_app_package/deploy-docker.html#Deploy-application-package-created-with-pyvespa the script fails with error:
RuntimeError: ["Command failed. No directory or zip file found: '/app/application'", '']
from vespa.package import ApplicationPackage
app_package = ApplicationPackage(name="my_package")
from vespa.package import Field
app_package.schema.add_fields(
    Field(name = "cord_uid", type = "string", indexing = ["attribute", "summary"]),
    Field(name = "title", type = "string", indexing = ["index", "summary"], index = "enable-bm25"),
    Field(name = "abstract", type = "string", indexing = ["index", "summary"], index = "enable-bm25")
)

from vespa.package import FieldSet
app_package.schema.add_field_set(
    FieldSet(name = "default", fields = ["title", "abstract"])
)

from vespa.package import RankProfile
app_package.schema.add_rank_profile(
    RankProfile(name = "bm25", first_phase = "bm25(title) + bm25(abstract)")
)

import os
from vespa.deployment import VespaDocker

disk_folder = "sample_application" # specify your desired absolute path here
vespa_docker = VespaDocker(
    port=8083,
    disk_folder=disk_folder
)

app = vespa_docker.deploy(
    application_package = app_package,
)

Waiting for configuration server.
Waiting for configuration server.
Waiting for configuration server.
Waiting for configuration server.
Waiting for configuration server.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_285/3496665072.py in <module>
      8 )
      9
---> 10 app = vespa_docker.deploy(
     11     application_package = app_package,
     12 )

/.env/lib/python3.8/site-packages/vespa/deployment.py in deploy(self, application_package)
    261         self.export_application_package(application_package=application_package)
    262
--> 263         return self._execute_deployment(
    264             application_name=application_package.name,
    265             disk_folder=self.disk_folder,

/.env/lib/python3.8/site-packages/vespa/deployment.py in _execute_deployment(self, application_name, disk_folder, container_memory, application_folder, application_package)
    232
    233         if not any(re.match("Generation: [0-9]+", line) for line in deployment_message):
--> 234             raise RuntimeError(deployment_message)
    235
    236         app = Vespa(

RuntimeError: ["Command failed. No directory or zip file found: '/app/application'", '']

Docker requires that you specify disk_folder absolute path. For example, on my machine (macOS) I need to specify disk_folder = "/Users/tmartins/sample_application" instead of disk_folder = "sample_application".

Thank you for the reply.
I tried as well with absolute path but again same error, e.g.
disk_folder =  "C://Users//COJ1LOD//projects//vespa//sample_application"
disk_folder =  "C:/Users/COJ1LOD/projects/vespa/sample_application"
disk_folder =  os.path.join("C:/Users/COJ1LOD/projects/vespa",
"sample_application" )
This happens when running on Windows, if I try on Linux then it works as
expected
Il giorno ven 24 set 2021 alle ore 12:21 Thiago G. Martins <
@.***> ha scritto:

Docker requires that you specify disk_folder absolute path. For example,
on my machine (macOS) I need to specify disk_folder =
"/Users/tmartins/sample_application" instead of disk_folder =
"sample_application".
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/vespa-engine/pyvespa/issues/211#issuecomment-926516118,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIRGNMDPXCGEL5OXVN7HTDTUDRGIXANCNFSM5EVOCMGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Nevermind my bd, I forgot to put the r before declaring the path.
disk_folder =  os.path.join(r"C:\Users\COJ1LOD\projects\vespa",
"sample_application")
Now it works as expected on Windows as well, thank you for your help and
sorry to bother you
Il giorno ven 24 set 2021 alle ore 14:49 Jason Cole @.***>
ha scritto:

Thank you for the reply.
I tried as well with absolute path but again same error, e.g.
disk_folder =  "C://Users//COJ1LOD//projects//vespa//sample_application"
disk_folder =  "C:/Users/COJ1LOD/projects/vespa/sample_application"
disk_folder =  os.path.join("C:/Users/COJ1LOD/projects/vespa",
"sample_application" )
This happens when running on Windows, if I try on Linux then it works as
expected
Il giorno ven 24 set 2021 alle ore 12:21 Thiago G. Martins <
@.***> ha scritto:

Docker requires that you specify disk_folder absolute path. For example,
on my machine (macOS) I need to specify disk_folder =
"/Users/tmartins/sample_application" instead of disk_folder =
"sample_application".
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/vespa-engine/pyvespa/issues/211#issuecomment-926516118,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIRGNMDPXCGEL5OXVN7HTDTUDRGIXANCNFSM5EVOCMGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

No problem @jjccooooll12, glad it worked.


Update color picker

Description (what's changed?)
Replaced the color picker with a more powerful one
Testing instructions

Add text and try to change color

Task linked: CU-86aydcam2 Make color picker with full range of colors


implement the Error interface

There are places where we return values such as for eaxmple (Address32, TransactionValidityError), but there may be other errors that could occur and we might want to propagate them as well. For this reason, we might consider implementing the Error interface for all types that resemble errors (TransactionValidityError, TokenError, etc.).
In certain places, we should check if the error is of the type in which we are interested.

as a next step we could remove Result types  (e.g. goscale.Result, DispatchResult, DispatchResultWithPostInfo)


Nuxt3 .server (island) components fail to render "Request Header Fields Too Large" if you pass a lot of data to props

Environment

Operating System: Darwin
Node Version:     v20.11.1
Nuxt Version:     3.12.4
CLI Version:      3.12.0
Nitro Version:    2.9.7
Package Manager<EMAIL_ADDRESS>Builder:          -
User Config:      vite, css, modules, $development, srcDir, serverDir, colorMode, app, runtimeConfig, components, compatibilityDate, hooks
Runtime Modules<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Build Modules:    -

Reproduction
/src/pages/news/[id].vue

<template>
  <ContentView :text="data.text" />
</template>

/src/components/content/ContentView.server.vue

<template>

</template>

server side: /server/api/news/[id].get.ts
export default defineEventHandler(async (event) => {
  try {
    return {
      text: 'Lorem ipsum dolor sit amet......', // For simple reproduce, it's enough to pass a text of approximately 50k chars.
      // ....other fields
    }
  } catch (error) {
    throw createError({ statusCode: 404, message: 'Material not found' })
  }
})

Describe the bug
I've a ContentView component that accepts article content from the API server, which I pass to v-html.
My task is to render ContentView ONLY on the server side. For this, I added the server suffix to component (ContentView.server.vue).
but the problem is that a lot of data is passed to the ContentView component into props.
<ContentView v-bind="data" />

and that's why I get error "431 Request Header Fields Too Large".
My browser request:
Request URL: http://localhost:3000/__nuxt_island/ContentView_KqN6hVg9ec.json?props=%7B%22text%22:%22Lorem+ipsum+dolor+sit+amet...... 50k chars
Request Method: GET
Status Code:  431 Request Header Fields Too Large
Remote Address: [::1]:3000

This probably not be a bug, but I need to do the component render only on the server side. Please any solutions.
Or, so that the request to ISLAND is not GET.
Thank you!
Additional context
No response
Logs
No response

This is not a bug but a limitation of the fact that server component props need to be passed in the URL. But you should probably refactor so the large data payload is fetched within the server component based on a smaller prop (e.g. an id), rather than fetching and sending a lot of data to the component.


query string not escaping '+'

1- create user with name='a+b'
2- fetch user with filter query set to name='a+b'; returns no items
3- fetch user with filter query set to name='a%2b'; returns user
I don't know how or where, but seems pretty clear somewhere it is doing ?name={name} in a query string, and it should be encoding but isn't

We aren't doing encoding ourselves anywhere, it's handled by the http library we use.
I can take a look when I have time, but I'm not sure what options we have available to us over the lower layer's encoding

I fixed this by doing urlencode() on values before passing them to the API

dude, you are doing string munging to generate query params
https://github.com/tableau/server-client-python/blob/b260ebcafec6e6ebe89597f13e1e18ed9ca55406/tableauserverclient/server/request_options.py#L95-L97
https://github.com/tableau/server-client-python/blob/master/tableauserverclient/server/request_options.py#L164
        self._append_view_filters(params)

        return "{0}?{1}".format(url, '&'.join(params))

that took me less than 5 minutes looking at the code

sorry for being a little short, trying to get tableau integration working has been my life for the past week or so

Right, but our http library itself encodes those values when it makes the request (I had a bug in the opposite direction recently, maybe this is actually related to that)
We can look at encoding it at the places you just mentioned, I just want to make sure nothing gets double encoded

makes sense -- maybe it is possible to pass the query string to HTTP as a key/value dict instead of doing the string construction yourself?

that seems to be the recommended way to use requests: https://requests.kennethreitz.org/en/master/user/quickstart/#passing-parameters-in-urls

https://github.com/tableau/server-client-python/blob/b260ebcafec6e6ebe89597f13e1e18ed9ca55406/tableauserverclient/server/endpoint/endpoint.py#L45 looking here it tries to manually smoosh up the URL, but instead it good just say parameters['params'] = request_object.get_query_params() or similar
there's no issue -- body data is data={...}; so even on a post request you can set params={...} and everything works fine

This works today.


Remove dependency on reactive extensions

For us it is rather problematic that this package depends on reactive extensions 3.0. We use ReactiveUI which depends on the older package/version. To be able to use WPFTabTip we need to include its source in our project which is rather unfortunate.

I am not entirely sure what are you proposing. You want to remove reference to reactive extensions and rewrite code that relies on Rx?

Yes, I'm currently investigating how easily this can be done.

You can rewrite code that relies on Rx with threads and locks, but I would seriously consider this only if your issue will gain some traction (If many people will have the same problem). Otherwise I would prefer to stick with Rx, even if that is a deal breaker for you, sorry.

I'm affected by the same issue.
What happened is this:
Microsoft published the reactive extensions library under the package id Rx-Main (and dependencies). A lot of other packages depend on these packages. One example is ReactiveUI.
Recently Microsoft decided to rename the package to System.Reactive (and dependencies). The DLL files included in this package are the same as in the Rx-Main package. But a slightly newer version.
This broke the upgrade path. I cannot update the Rx-Main package so it works with WpfTabTip and I cannot install the System.Reactive package because it conflicts with the Rx-Main packages. (The DLL files in the package still have the same name, so I get duplicate references and files get overwritten during build).
What I did for now was to include the entire WpfTabTip source code in my application and change it to reference the Rx-Main package instead of the System.Reactive package. Works perfectly. 😄

I've logged an issue with System.Reactive, unfortunately they can't fix it on their end: https://github.com/Reactive-Extensions/Rx.NET/issues/416

@roy-t, what about ReactiveUI developers, is there any hope that they will move to a newer version of Rx anytime soon? Or do you have problems not connected to ReactiveUI?

I believe a new version of ReactiveUI is in the works. But I don't think there is a release date has been set. I do know they plan on using the new nuget package for System.Reactive. So I guess this problem will solve itself in the end.


Add display reset command.

Will be useful after view event works

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 0% with 8 lines in your changes missing coverage. Please review.

Files with missing lines
Patch %
Lines

...dress/logic/commands/ResetDisplayListsCommand.java
0.00%
4 Missing :warning:

...rc/main/java/seedu/address/model/ModelManager.java
0.00%
3 Missing :warning:

.../seedu/address/logic/parser/AddressBookParser.java
0.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Files with missing lines
Coverage Δ
Complexity Δ

src/main/java/seedu/address/model/Model.java
0.00% <ø> (-100.00%)
0.00 <0.00> (-1.00)

.../seedu/address/logic/parser/AddressBookParser.java
72.22% <0.00%> (-27.78%)
6.00 <0.00> (-6.00)

...rc/main/java/seedu/address/model/ModelManager.java
41.26% <0.00%> (-58.74%)
12.00 <0.00> (-11.00)

...dress/logic/commands/ResetDisplayListsCommand.java
0.00% <0.00%> (ø)
0.00 <0.00> (?)

... and 23 files with indirect coverage changes


treecbh::get_CBH issue

Dear Author,
I encountered the following error when running treecbh::get_CBH using the example data provided:
”A_CBH <- treecbh::get_CBH(las_l, cbh_ONLY = 1, outdir1 = outdi1, outdir2 = outdi2, cc_dir = cc_dir)“
Error message
Error in group_by():
! Must group by variables found in .data.
✖ Column intermediate_segs is not found.
Run rlang::last_trace() to see where the error occurred.
Could you please advise on how to resolve this issue?
Thank you very much.

Dear Theo,
the issue is connected to the non-sufficient point density of the example data (input tree point clouds).
It is recommended to check how well the point cloud captures the tree structure before applying get_CBH().
#> Let's proceed with the five trees that have sufficient points
las_l <- its_l[c(7,9,72,78,131)]
Here you can visually check the inpout tree point cloud of your selection:
plot(readLAS(las_l[1]), bg = "white", size = 5, axis = T)
...
plot(readLAS(las_l[5]), bg = "white", size = 5, axis = T)
A_CBH <- treecbh::get_CBH(las_l,
# run tree isolation and cbh detection
cbh_ONLY = 1,
# its point cloud directory
outdir1 = outdi1,
# point cloud segments (stem plus first leaved branch) directory
outdir2 = outdi2,
cc_dir = cc_dir)
A_CBH
Note:

This is  a dummy example, not really suitable for the tool
The k-Means-based clustering has been changed to visual CBH estimation.
See under "Parameters of treecbh" section:
https://academic.oup.com/forestry/advance-article/doi/10.1093/forestry/cpae044/7746722

I have to update GitHub page as well showing the visual CBH estimation.
Take care and thanks,
Gergo

Dear author
Thank you for your prompt reply. This problem seems to be unrelated to the point density quality of the single tree point cloud data.
I used the TLS single tree point cloud data set mentioned in the paper https://academic.oup.com/forestry/advance-article/doi/10.1093/forestry/cpae044/7746722 (https://www.forestgeo.si.edu/sites/europe/wytham-woods) to perform this operation, and the problem still occurred.

It seems you have not decimated the input point cloud. Hight point density is no use, as it described in the paper (Point density analysis). This kind of data needs treeiso reparameterisation, refer to https://www.mdpi.com/2072-4292/14/23/6116. I recommend to decimate the input point cloud to a point density (100-120 pts/m^2) so that it nicely repersents the tree structure.
For point cloud decimation use the lidR::decimate_points() command, I used the homogenize algorithm, but you might try random or the rest.
https://rdrr.io/cran/lidR/man/decimate_points.html


Fix - Cell Reordering in IGListBindingSectionController (viewModels array not being updated)

See #1262 (cell reordering using a binding section controller caused problems)

Changes in this pull request

Added method override in IGListBindingSectionController to update the internal viewModels array after a cell has been moved
Subclasses of IGListBindingSectionController require to call super

Issue fixed: #1262
I've added a test for this change. Is this enough? Or should I add more corner cases or something?
Checklist
Some tests fail, but they do seem unrelated. I don't know if this is normal. I haven't changed anything regarding those failing tests.

[ ] All tests pass. Demo project builds and runs.
[x] I added tests, an experiment, or detailed why my change isn't tested.
[x] I added an entry to the CHANGELOG.md for any breaking changes, enhancements, or bug fixes.
[x] I have reviewed the contributing guide

Any update on when this will be merged? Is there anything I can do to help?

@d3mueller thanks for sending this out. @rnystrom Any chance you can review this again so it can be merged.  The build failures appear unrelated to the actual changes in the PR.


NixOS SD card image reproduction Q&A

@bflaggs @ZishuoYang Let's use this issue to track all problems regarding the reproduction of SD card images for our RPi 3 and 4's.

I've restored the dropbox link listed in the README file.
To differentiate between RPi 3 and 4 image, just take a look at the hostname of the Pi: RPi 4 has a hostname of LabPi4, RPi 3 LabPi3.

Before I leave, I left a working Pi 3 AND Pi 4. So if you just need two Pi's for development, you don't need to install one from scratch.
Also, I think it is fine to do testing with a Raspbian, provided that you don't override existing NixOS SD cards.
Here's a quick reference to reconfigure an existing NixOS setup:
cd /etc/nixos
git pull origin rpi4  # replace 'rpi4' with 'rpi3' if you are on 'rpi3'
sudo nixos-rebuild switch

If there's no error, the Pi is configured correctly. Post error messages here if you encounter problems.

The updated configuration makes /dev/gpiomem writable for all users in the gpio group (just like what Raspbian does). If you can't reconfigure it, current configuration is still workable: Just prepend a sudo when you do the testing:
sudo ./FireAlarm.py 4  # '4' is the GPIO pin number that connects to the fire alarm

@bflaggs If you can list the confusing parts in the README, I can try to explain it a bit better.

@yipengsun there was an error when trying to rebuild the Pi 3 system after pulling the repo rpi3. The output to the terminal from the command sudo nixos-rebuild switch is included in the file rebuild_errors.txt in the repo rpi3.
It should also be noted that we were unable to use the Pi 4 because I do not believe that the Pi 4 is properly configured. When I start up the Pi 4, NixOS will start however there is no login screen that comes up and I am immediately logged in. The terminal states something about root not being setup so automatic login was established. Because of this the hostname on the Pi 4 is nixos NOT LabPi4 as it should be.

The error log is useful. I'll fix it soon. However, please attach the error log to the issue, instead of push it to a branch.

As for reconfiguring another Pi 3, the steps are:

Download one of my pre-approved images, then dd it to a SD card.
Clone this repo to /etc/nixos on the SD card, following instructions in README.
Insert the SD card to a Pi 4, now the hostname should be nixos
Configure a temporary password for user nixos (This is the default user for the installation media, and does not exist in the final setup) with passwd then type in the password of your choice.
Run sudo nixos-rebuild switch on the Pi 4.
Go to /etc/nixos in the Pi 4, switch to pi3 branch
sudo nixos-rebuild switch again
Remove the SD card from Pi 4, insert it into a Pi 3.

Another thing for error log: If you have a SSH connection to the Pi, then copy the error log should be easy, otherwise I guess it's fine to just push the log to a branch to minimize trouble.
Later I/we can download the file via github web interface and attach the file here. Then I'll do a force push to remove that commit. But it is fine to use a branch as a temporary storage.

@bflaggs I've pushed a "fix" to both branches. Will you test if they work?
Also, I've removed your commit to rpi3 branch and attached the file in your post. If you want to pull the rpi3 branch, I'd suggest you to do the following:
git reset --hard HEAD~1  # on rpi3 branch
git pull origin rpi3

@yipengsun I will check if they work when I come in tomorrow afternoon, which will be around 1pm EST.
As for the error logging, I do not have SSH connection to the Pi which is why we pushed directly to the rpi3 branch. So, unfortunately, we will have to continue pushing the error log files and then rearranging the files. Hopefully no more errors arise so that we will not need to do this anymore.

For building new images, internet connection is required (to download source code of linux kernel). How'd you configure internet access on pis?

I was able to get internet access on the pis by sharing the internet from my computer. I would bridge the network connections for my laptop's WiFi and my laptop's ethernet port and then I would connect to the pi via and ethernet cord. Although doing so takes my laptop's internet connection away.

@yipengsun the fix now allows a rebuild of the system without errors. Although we still receive all of the warnings that were included in the above log file.
Now, after pulling the updates to rpi3 and then rebuilding the system using sudo nixos-rebuild switch, the python scripts WaterAlarm.py and FireAlarm.py will not run as root. When running as root they both error and say that there is no module named rpi. A log of the terminal output is attached here:
wateralarm_errorlog.txt
I can still run WaterAlarm.py without using the command sudo and it seems to tun okay (I have not tested if it can detect water) although running FireAlarm.py without using the sudo command will also result in the error of no module named rpi.

I have run the command python3 as root and not as root and have determined that the system paths (the output paths from the Python command sys.path) are different when running the command sudo python3 compared to the command python3.
When I do not run Python 3 as root then there are more paths within the system (more paths are output when I issue the python command sys.path), I believe that the module RPi is within one of these extra paths as I am able to import module RPi when I am NOT running Python 3 as root. Although when I DO run Python 3 as root I am not able to import this module.
This ModuleNotFoundError for importing RPi causes the ModuleNotFoundError for rpi (the repo including the python scripts) because the import error for RPi occurs within the snippet of code below:
try:
    from rpi.burnin.WaterAlarm import WaterAlarm
except Exception:
    sys.path.insert(0, "..")
    from rpi.burnin.WaterAlarm import WaterAlarm

The Exception is raised and then WaterAlarm is imported properly although within WaterAlarm we import RPi, which when running as root can not be found. So the codes error when running as root.
I'm not sure how to fix this but I'm very confident that this is the problem. As I said in the post above I will continue debugging without running either script as root.

Now that I think of it, root probably can't see the Python libraries to due to how NixOS works. I think now if Fire and Water alarm scripts work as a user, then all is good. If not, I can update configuration file of NixOS so that all libraries are installed globally.

@yipengsun please clarify the instructions regarding mounting partitions (in the last command line, which partition should we mount?) :
In Linux, suppose we have a sdcard mmcblk0 with 2 partitions mmcblk0p1 and mmcblk0p1, then we should write to mmcblk0, not either of the partitions, with the following command:
sudo dd if=<nixos.img> of=/dev/mmcblk0
On first boot, the rootfs will be automatically resized to use all available spaces of the SD card.
After the dd command is completed, mount the /dev/mmcblk0p2 to /mnt:
sudo mount /dev/mmcblk0p1 /mnt

The latter of the two (p1). Apparently I can't count properly but you should know me well enough by now to not trust me counting :-)
Lemme fix that in the README.md as well.

@yipengsun when configuring a microSD card on the rpi 4, after running the command sudo nixos-rebuild switch and rebooting the pi, the user lhcb is not created. On the rpi 4 there is still no actual login and the user is nixos@nixos. I also listed the contents of the file /etc/passwd and there was no user with the name lhcb.
It is worth mentioning that when I ran the command sudo mount /dev/mmcblk0p1 /boot I received an error stating mount: /boot: mount point does not exist.. To correct this error I just made the /boot directory using sudo privileges.
Another error I received that might be important is after I ran the command sudo nixos-rebuild switch, the command directly after mounting the boot partition, I received the error that the file /etc/nixos/configuration.nix did not exist. I also ignored this error because in your README file it says to ignore warnings and errors that occur during that step.

It is worth mentioning that when I ran the command sudo mount /dev/mmcblk0p1 /boot I received an error stating mount: /boot: mount point does not exist.

This is an indication that the root partition is not set up correctly. In my last post, I claimed that you should do this:
sudo mount /dev/mmcblk0p1 /mnt

Under the assumption that p1 is the second partition. That is wrong: My initial recollection told me that partitions should start with 0. When I wake up today, I think I was wrong: partition should start with 1! Sorry!. You can confirm this by checking the partitioning scheme with lsblk.
Try the following mount (old instruction):
sudo mount /dev/mmcblk0p2 /mnt

After that mount, you can cd into /mnt and folder boot should be there.
If not, then the disk image is not properly written to the SD card. If you are using a pi 4 to recreate another SD card image with a USB card reader, You need to be careful about the path to the disk: The USB card reader could (again, check with lsblk) map the SD card to something like /dev/sda, not /dev/mmcblkX (This depends on the hardware).

Another error I received that might be important is after I ran the command sudo nixos-rebuild switch, the command directly after mounting the boot partition, I received the error that the file /etc/nixos/configuration.nix did not exist. I also ignored this error because in your README file it says to ignore warnings and errors that occur during that step.

This error is indeed too serious to ignore. Follow my troubleshooting guide listed in the above paragraph.

Given the difficulties recreating SD cards, If we still have two working NixOS pi's (one for pi3, one for pi4), I think you can use Raspbian for the rest of the pi for easier setup. The NixOS systems can be reserved for burn-in servers only.

It's too bad that I can't SSH into my computer on my desktop, otherwise I can do these things remotely.

Another point: One of my initial motivation to switch to NixOS is: People keep re-installing Raspbians on my old Arch linux rpi SD cards, which means all my previous configurations were lost. The good thing about NixOS is that I (not everybody) can recreate my old setup with ease so that the rest of us can use Raspbians freely.
I think this is the correct way to go: Install Raspbians on the new SD cards while keeping the old NixOS ones for burn-in servers. You probably still need to do some debugging on NixOS, but you don't need to making new NixOS images.

I do believe that when we attach the USB card reader it does have a path of /dev/sda not /dev/mmcblkX. Zishuo would know more about how the mounting part of this worked as he was the one who performed this part yesterday. I can also take a look and see if this is the case.
As for what you think is the correct way to go I will check with Manuel and Zishuo when they arrive today and see if they agree. However, when you left you said that there was a working/configured rpi 4 although the only rpi 4 that we could find was one that did not seem to be fully setup. I briefly mentioned this in the 2nd paragraph of this post https://github.com/umd-lhcb/nixos-config-rpi/issues/1#issuecomment-582481249.

I see. It's possible that I forgot to fully configure pi 4, as I was using my own SD card on pi 4 and may forgot fully configure the SD card that was installed.
You could use the same instruction guide to re-configure that pi 4. I guess you should mount mmcblk2/sdX2 to /mnt and clone the content of this project to /mnt/etc/nixos.
It would be better if we can schedule a meeting and you can show me your procedure.

I'm planning to talk to Manuel 10am EST. @bflaggs maybe we can talk afterwards?

That works for me, did you want to talk via slack?

Also I have the procedure typed out in my notes up to the part labeled System Configuration in your README. Again Zishuo figured out the way to do this and I just typed what he told me and how I understood it although I could send you what I have.

This was everything that I did to fully configure a pi 4 with NixOS and then fully configure a pi 3 with NixOS.
I was able to fully configure a pi 4 with NixOS by following the following steps:
Follow instructions below to install the OS.

Save Yipeng's image file to USB memory stick.
Insert memory stick to rpi3 with configured a microSD card in microSD slot.
Save Yipeng's image file from USB stick to configured microSD card (just make a new directory under the /home directory called image_file_to_install or something and save it here).
Unzip the image file (because it should be zipped as a .bz2 file) using the command bzip2 -dk <name_of_bz2_image_file>.
Insert USB converter stick with unconfigured microSD card in the SD slot (Check the name of this new microSD card by doing ls /dev before and after inserting this USB converter. NOTE: Typically the name of the unconfigured microSD card will be sda with two partitions sda1 and sda2).
If microSD card has name sda then use command sudo dd if=<path_to_nixos.img> of=/dev/sda  in order to setup NixOS on the unconfigured mircoSD card (NOTE: if microSD card does not have name sda then replace sda in the command with whatever the name of the microSD card appears to be in the /dev directory)
Mount the microSD card using command sudo mount /dev/sda2 /mnt.
Clone the rpi4 NixOS configuration from GitHub (NOTE: The mkdir and all the subsequent git commands to be executed in this step must be run as root, this step is outlined here: https://github.com/umd-lhcb/nixos-config-rpi).
Unmount the microSD card using command sudo umount /mnt.
Unplug microSD card from USB on rpi4 and plug into microSD slot on rpi4.

Follow instructions below to set up the OS.

On first boot of the pi set the password with command sudo passwd root (enter any password you like, this password does not matter and will be erased soon).
Switch to root user using command su.
Issue command ls / and look to see if directory /boot exists, if so move to step 5.
Create the /boot directory using the command sudo mkdir /boot.
Mount the boot partition using the command sudo mount /dev/mmcblk0p1 /boot (NOTE: Here we mount using mmcblk0p1 because the microSD card is now in the mircroSD slot not in the USB slot. Also notice here we mount using partition, i.e. 0p1 , because this partition is where the contents which need to go into the /boot directory are saved.).
Rebuild the system using sudo nixos-rebuild switch (there will be errors at this step which will be okay because the SD card will not be fully setup yet HOWEVER, if you receive an error stating the etc/nixos/configuration.nix file could not be found and the build starts erroring A LOT then there is a bigger error. Most likely something was not setup correctly in one of the previous steps such as either branch rpi4 was not cloned correctly from GitHub or the wrong microSD partition was mounted).
 (NOTE: You do not need to unmount the partition before powering off as the pi will automatically unmount the partition once it receives the signal to power off.).
On reboot perform steps 2-7 again and exactly as they are described above (if /boot directory does not exist and you need to remake it after rebooting then you did something wrong along the way).
Now the user lhcb should have been made, perform command less /etc/passwd in order to check the users on the system. Make sure that lhcb is an actual user and the move to the next step. If there is no user with this name then you have done something wrong, reboot and try steps 2-7 again.
Again reboot the system following the actions outlined in step 7.
On reboot you should be asked for the login of LabPi4 (only if you are on the pi 4), login with user lhcb where the password is the same as the username. The reboot will also state something about zshell not being completely configured, when it asks for an input you can read the answers it expects and do one of those commands. I performed command 0 stating just to put a comment in the zshell file so that this warning would not pop-up again.
If everything looks good/normal then we can add the official unstable package using command sudo nix-channel --add --add https://nixos.org/channels/nixos-unstable nixos.
Update the system again using the command sudo nixos-rebuild switch --upgrade.
If upgrading the system works then you can delete the old unnecessary files with command sudo nix-collect-garbage -d.

Note: The above configuration is for setting up the microSD card for a pi 4.
To set up the microSD card for a pi 3 then follow the steps exactly as outlined above then perform the following steps afterwards:

Make sure that the microSD card is plugged into a pi 4 and is fully setup to work correctly with NixOS.
Change directory into where the nixos-config-rpi repo is set using the command cd /etc/nixos.
Run the commands sudo git fetch origin then sudo git checkout rpi3 in order to switch to branch rpi3.
Remove all the files in the /boot directory using the command sudo rm -rf /boot/*. This is because the boot directory is too small to hold everything it needs.
Rebuild the system using command sudo nixos-rebuild switch. NOTE: Yipeng says in his README on branch rpi3 that this step will take~3 hours to complete because the rpi3 kernel needs to manually compile however, if the commands are run in this exact way where you setup the rpi4 kernel before the rpi3 kernel then this rebuilding only takes a few minutes.
Now poweroff the pi 4 using sudo poweroff then plug the microSD card into a pi 3 and everything should be configured.


Codenet subsets rework

This PR adds the 2 subsets of the CodeNet dataset. Corresponding changes are made for notebooks and references.

cc: @ckadner
I signed off the commits every time but the CI check still fails.


Support [Exclude tags

Hi!
The completion engine should hide APIs that specified by the [Exclude metadata tags.
A typical example is the Stage class:
[Exclude(name="y",kind="property")]
[Exclude(name="x",kind="property")]
[Exclude(name="visible",kind="property")]
[Exclude(name="transform",kind="property")]
[Exclude(name="tabIndex",kind="property")]
[Exclude(name="tabEnabled",kind="property")]
[Exclude(name="scrollRect",kind="property")]
[Exclude(name="scaleY",kind="property")]
[Exclude(name="scaleX",kind="property")]
[Exclude(name="scale9Grid",kind="property")]
[Exclude(name="rotation",kind="property")]
[Exclude(name="opaqueBackground",kind="property")]
[Exclude(name="name",kind="property")]
[Exclude(name="mouseEnabled",kind="property")]
[Exclude(name="mask",kind="property")]
[Exclude(name="loaderInfo",kind="property")]
[Exclude(name="focusRect",kind="property")]
[Exclude(name="filters",kind="property")]
[Exclude(name="contextMenu",kind="property")]
[Exclude(name="cacheAsBitmap",kind="property")]
[Exclude(name="blendMode",kind="property")]
[Exclude(name="alpha",kind="property")]
[Exclude(name="accessibilityProperties",kind="property")]
[Exclude(name="accessibilityImplementation",kind="property")]

The kind can be property|method|event|style|effect.
A real example:
[Exclude(name="hasEventListener",	kind="method")]

/**
 * deprecated: Use hasListener or hasEvent
 */
override public function hasEventListener(type:String):Boolean
{
	...
}

public function hasEvent(type:String):Boolean
{
	...
}

public function hasListener(listener:Function):Boolean
{
	...
}

Thanks!

[Exclude] metadata is supported.
https://github.com/BowlerHatLLC/vscode-as3mxml/blob/199a9cbb406925140488c1d83b62bd2a50303d76/language-server/src/main/java/com/as3mxml/vscode/providers/CompletionProvider.java#L1845-L1857
You need to add [Exclude] metadata above the class definition.
[Exclude(name="hasEventListener",	kind="method")]

class MyClass {}

You need to add [Exclude] metadata above the class definition.

I see it worked with the class that has [Exclude] metadata, but it does not work with sub-classes.
Thanks!

Hi!
I've tried the latest snapshot but I still see the excluded members in sub-classes.

Could you recheck this?
Thank you!

Could you recheck this?

In my test project, it appears to be working correctly. I can add [Exclude] to any class in the inheritance chain, and the member is correctly excluded from completion.

There was a problem when I installed the latest extension snapshot. It show me a progress bar for a long time (over a minute) until it finished. Maybe that was the cause. I re-built and & re-installed the vsix 2 times until it did not show the progress bar anymore, restart vscode, then it's working fine now.
Thank you!


Add research done with Fish to the README

Kicking this down the road to Barracuda as I'm waiting for Bob to get us the list and I don't want it to hold up the release.


HmIP-SWSD on piVCCU3: Siren control via hassio

Hi! I'm just migrating from FHEM to hass.io. For testing purposes I set up piVCCU3 with an HmIP-RFUSB and one paired HmIP-SWSD. Is there a way to forward the HmIP-SWSD's siren to hass.io as a switch device to turn it on and off from within hass.io?
Another thing: I get a few warnings in hass.io:

Setup of homematic is taking over 10 seconds.
<Fault -5: 'Unknown Parameter value for value key: LOW_BAT'>
<Fault -5: 'Unknown Parameter value for value key: RSSI_DEVICE'>
<Fault -5: 'Unknown Parameter value for value key: SMOKE_DETECTOR_ALARM_STATUS'>

Is this something to worry about?

The correct way to manually trigger the alarm is to use the homematic.set_device_value service with a payload like the following:
address: 001122AABBCC
channel: 1
param: SMOKE_DETECTOR_COMMAND
value: 2

The value 2 turns the alarm on, 1 turns it of. This is for the intrusion alarm. To turn off the smoke alarm you have to use the value 0. The value 3 would start a smoke alarm test, but I don't know if this would automatically be forwarded to other devices, and how long it would stay active. Feel free to report your observation.
With that knowledge you can create an imput_select to display the options you want to use, and then use the selected option within an automation the control the siren.

I've tried 0, 1 and 2 which do work. As a feature request, I thought it might be easier if the interface would provide a dedicated switch device for turning the alarm on and off, like the ZigBee Smoke detectors with controllable siren do.
What about the warnings? Are they normal?

This should now be possible with the new integration.


CI: add -v and *-musl to the chown path

Resolves #227.

chown: cannot access '/usr/local/*-musl': No such file or directory

@matthew-levan said this will happen whenever there's a cache miss, since the directory won't be present.  I'm starting to wonder whether the ideal way to fix this might be to figure out how to avoid using sudo when building musl libc.  That way we should't have to chown anything.
Alternatively, we could investigate in more detail why it is that sudo chown /usr/local is taking ten minutes -- maybe there's something we could do to make sure that folder doesn't have so much stuff in it.

If I remember correctly, the problem is GitHub Actions can't populate /usr/local/*-musl from the cache unless /usr/local is owned by runner, not root. The musl-libc toolchain needs to be installed in a configuration-independent directory so that Bazel can specify absolute paths to it that work on all platforms and user configurations. All directories that meet that criteria that I can think of are owned by root, so avoiding sudo may be unavoidable. A work-around might be to chown $USER /usr/local && (chown -R $USER /usr/local*-musl || true).

@mcevoypeter's workaround works great, reducing the time for the chown /usr/local/ step from ~10 minutes to 0 (!) seconds^1. The command used is sudo chown $(whoami) /usr/local && (chown -R $USER /usr/local/*-musl || true).


Keplr Suggest Chain for unsupported chains

Keplr only supports a few chains out of the box. I get an error There is no chain info for cheqd-mainnet-1 for Cheqd for example. We need to use the Suggest Chain feature to hook this up. Info can come from Chain Registry, and cosmos.directory APIs.
Adding issue for my tracking but this will hold up #76 and others for now. @Galadrin I think we chatted about this too.

Not sure what issue you're referring to here.  I was able to add cheqd to Keplr just fine when I started a local version of restake.
Just the denoms looked broken

#87 This is covered in my last commit but we will need to make some changes to networks.json @tombeynon

Base denom (Ex.: ATOM vs minimal denom: uatom)
coinGeckoId
coinType (To support other types than 118 that is currently hardcoded

@pwnfoo if you already know Cheqd network in your Keplr, you dont see the issue.
on a fresh Keplr install, you will need to add the network, and the SuggestChain is mandatory

Fantastic work 💥
Base denom we can get from chain registry
Coin gecko again in CR
Coin type we can just set in networks.json when we identify a network needs it.
Might separate your Suggest Chain work from #87 so we can merge it separately, would be great to get that in sooner

@tombeynon #91  Here is the clean branch, I will need to add the 2 values from CR

Fantastic work 💥
Base denom we can get from chain registry Coin gecko again in CR Coin type we can just set in networks.json when we identify a network needs it.
Might separate your Suggest Chain work from #87 so we can merge it separately, would be great to get that in sooner

These have been done in my new PR #91

#123  - Merging as we speak


[Event Request] codeunit 5763 "Whse.-Post Shipment" - OnInitSourceDocumentHeaderOnBeforeTransferHeaderInit

We would like to add a new event OnInitSourceDocumentHeaderOnBeforeTransferHeaderInit in the following trigger. Same way as it has been done in codeunit 5760 "Whse.-Post Receipt"
local procedure InitSourceDocumentHeader()
...
DATABASE::"Transfer Line":
begin
//A108F0000.Event18.03.ns
IsHandled := false;
OnInitSourceDocumentHeaderOnBeforeTransferHeaderInit(TransHeader, WhseShptHeader, WhseShptLine, ValidatePostingDate, ModifyHeader, IsHandled);
if not IsHandled then
//A108F0000.Event18.03.ne
if not IsHandled then
if (TransHeader."Posting Date" = 0D) or
(TransHeader."Posting Date" <> WhseShptHeader."Posting Date")
then begin
TransHeader.CalledFromWarehouse(true);
TransHeader.Validate("Posting Date", WhseShptHeader."Posting Date");
ModifyHeader := true;
end;
Event Signature:
[IntegrationEvent(false, false)]
local procedure OnInitSourceDocumentHeaderOnBeforeTransferHeaderInit(var TransferHeader: Record "Transfer Header"; var WarehouseShipmentHeader: Record "Warehouse Shipment Header"; var WarehouseShipmentLine: Record "Warehouse Shipment Line"; var ModifyHeader: Boolean; var ValidatePostingDate: Boolean; var IsHandled: Boolean);
begin
    //A108F0000.Event18.03
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Intermittent Chrome Device Emulation issues on Mac. Window won't resize to mobile viewport size.

What is your Test Scenario?

Chrome device emulation feature works well 90% of the time for me but in some occasions, it fails to resize the browser window to the specified device viewport size. This issue is seen on Mac. For e.g, if I run this command testcafe 'chrome:emulation:device=iphone 6/7/8' test.js, it resizes the browser window to iphone 6/7/8 device viewport size. But sometimes TestCafe fails to resize the window. It stays in the desktop window size and won't resize. To make it work again, I have to try running different device modes or commands like this testcafe 'chrome:emulation:device=pixel 2' test.js or testcafe "chrome:emulation:width=100;height=200;mobile=true;orientation=vertical;touch=true" tests/sample-fixture.js to make it work again.
Not sure if we have any issue in the testcafe-browser-toolsresize util on Mac's. Anyone else seen or come across this issue? Note: When trying to reproduce this issue, please quit execution during test run and try again. I wonder it has something to do with previous test execution not closed properly and TestCafe trying to run again which messes some things.
What is the Current behavior?

Chrome device emulation not working all the time on Mac
What is the Expected behavior?

Chrome device emulation should work without any issues.
What is your web application and your TestCafe test code?

No specific tests to provide.

Execute this command...
See the error...

Your Environment details:

testcafe version:                    v1.1.4
node.js version:                     v8.11.1
command-line arguments:   testcafe 'chrome:emulation:device=iphone 6/7/8' test.js
browser name and version:  Chrome 74
platform and version:           macOS 10.14
other:

I was not able to reproduce the issue.
Besides, I tried to run tests concurrently and break test execution as you described, however, in all cases it works as expected. Does this problem appear only with TestCafe 1.1.4 version or with he previous one as well?
Do you use some custom user profile?
I would appreciate it if you share some additional information about the issue, e.g. video which demonstrates the resizing problem

@AlexKamaev This happens with the previous version v1.1.3 too. Attached 2 video dropbox links here to show the issue.
First is 'Not_working.mov' version  - Not_working.mov
Second is 'Working_fine.mov'. - Working_fine.mov
In the 'Not_working' video, you can see the window is not getting resized when device emulation is specified. The same command works fine later (Workaround that I did to make it work. I switched to another git branch & then back to develop branch).

I discussed the issue with the team and we decided that it can be related to the https://github.com/DevExpress/testcafe/issues/3399 problem, since you mentioned the situation when previous test execution was not executed correctly.
Please try to call the killall "Google Chrome" command each time you execute TestCafe.
Also I added some console.log and debugger calls to the code related to the issue. Could you please check this and share results?
testcafe.zip

@AlexKamaev Tried killall "Google Chrome" command for each run. It's not helping resolve this issue (browser still opens in normal window size). I tried the provided testcafe.zip and ran some tests. Not seeing any console.log in terminal. See screenshot below.

But if I insert await t.resizeWindow( 300, 300 ) in the test, then I see console.log in terminal. Am I doing something wrong or missing some instructionsn?

It looks like running the killall "Google Chrome" command is not helping. Could you please execute the following code instead:
ps -e | grep [Cc]hrome
ls -l $TMPDIR | grep testcafe
ls -l $TMPDIR/testcafe
rm -rf $TMPDIR/testcafe

(it lists information about Chrome processes, testcafe temporary directory, it's contents, and deletes these contents).

@arubtsov @AlexKamaev I executed the provided commands. Still no luck. It's not working either. Browser still opens in normal window size. I think it's ignoring the chrome emulation device type. Could it be some command line parsing bug when it's trying to read the device type?

@sijosyn
At the moment, we don't know whether or not this issue is related to the command line parsing algorithm.
It seems that testcafe cannot connect to the Chrome debug protocol for some reason. Perhaps some error occurs in this code: https://github.com/DevExpress/testcafe/blob/9c96672c9c6a7fb50658a4c57f1e17ce645cd3b6/src/browser/provider/built-in/dedicated/chrome/cdp.js#L65
I've prepared another archive with some logs which cover the described code. Please check it and redirect the output in a separate file, i.e. : testcafe "chrome:emulation:device=Pixel 2" test.js > report.txt
The output should look as follows:
open browser: _uR1BSX http://<IP_ADDRESS>:54110/browser/connect/_uR1BSX
create cdp client
get active tab 54112 - _uR1BSX
remote chrome
remote chrome connected
set emulation
 Running tests in:
 - Chrome 74.0.3729 / Windows 10.0.0

 fixture
 √ test

 1 passed (1s)

testcafe.zip

@AlexKamaev See the latest results below. I see error on connecting to cdp in the logs.
open browser: 2PcyHRS http://<IP_ADDRESS>:52688/browser/connect/2PcyHRS
create cdp client
get active tab 52698 - 2PcyHRS
error on connecting to cdp
 Running tests in:
 - Chrome 74.0.3729 / Mac OS X 10.14.2

 fixture
 ✓ test

 1 passed (2s)

It definitely looks like a problem with Chrome Debug Protocol.
Here is the problematic part of code:
https://github.com/DevExpress/testcafe/blob/9c96672c9c6a7fb50658a4c57f1e17ce645cd3b6/src/browser/provider/built-in/dedicated/chrome/cdp.js#L6
We have not faced any similar issue, so we need some additional time to research it.
You can try to add the cdpPort parameter to your emulation arguments as described in this article: https://devexpress.github.io/testcafe/documentation/using-testcafe/common-concepts/browsers/using-chrome-device-emulation.html
You can also try examples from the https://github.com/cyrus-and/chrome-remote-interface repository, which is the core part of our emulation mechanism.

Thanks @AlexKamaev. I added the cdpPort parameter to emulation argument like this (testcafe "chrome:emulation:device=pixel 2;cdpPort=9223" abc.js > report.txt). Still no luck. Got same result (see log below). I'll look into examples provided in the chrome-remote-interface repo and will let you know.
open browser: mAj_GWO http://<IP_ADDRESS>:57447/browser/connect/mAj_GWO
create cdp client
get active tab 9223 - mAj_GWO
error on connecting to cdp
 Running tests in:
 - Chrome 74.0.3729 / Mac OS X 10.14.2

 fixture
 ✓ test

 1 passed (1s)

@AlexKamaev I found the reason why this is happening on my machine. I'm using hostfile manipulation tool called hostile to add a bunch of ip addresses in the hostfile (overide DNS) before running the tests to hit URL's(UAT/Staging machines) that are behind firewall/non-public domain.
When writing to the hostfile, it deleted the following required localhost block and that was causing this issue. Adding the localhost entry in the hostfile resolved this issue. I'll close this issue for now.
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
<IP_ADDRESS>	localhost
<IP_ADDRESS>	broadcasthost
::1             localhost

We are happy to hear that you have found the cause of the issue. Thank you for sharing this information with us.


Add support for Django 1.10

Coverage decreased (-0.8%) to 91.882% when pulling 5214381a502a34956dccc89e1fa62b3bdd42f23c on feature/django110 into d17539998a5f638cec9f88bc269271fd9b37124b on master.

Coverage decreased (-0.1%) to 92.509% when pulling 05d66fdcac52c7a48e7b83770349460f202675be on feature/django110 into d17539998a5f638cec9f88bc269271fd9b37124b on master.

Coverage increased (+0.6%) to 93.208% when pulling ef9f7d78d545253249570994fc1f31ed14d441d5 on feature/django110 into d17539998a5f638cec9f88bc269271fd9b37124b on master.

@czpython could you review this?

@czpython @stefanfoulis I'd like a review from one of you

@czpython @stefanfoulis I'd like a review from one of you


Fixing the network graph

Coverage remained the same at 96.046% when pulling e420fd3ea201402927b2c351d15fa8c2fc541191 on development into a8bf13d3caebeec719b396befedaa3a530f3f71c on master.


Genotype strategy

What is the recommended way to obtain genotype information for assigning donor id that is also cost effective?
How does using genotype array compare with using variants called on bulk rna seq or bulk exome seq?

Hi, as you may already notice that vireo outputs the estimated genotype for each donor, which you can use to identify the sample id.
I'm not familiar with the cost of these sequencing platforms, but all of the three (genotype array, bulk RNA-seq and bulk exome-seq) work very well in my experience. I believe other 'omics data, e.g., ATAC-seq, CHiP-seq, will do it well too.


Compile crates with cargo build instead of cargo deb

The cargo deb subcommand is not fully compatible with workspaces, so
using it for deb package creation only provides us with more consistent
results.
Signed-off-by: Ryan Beck-Buysse<EMAIL_ADDRESS>

This is similar to a recently merged PR in Splinter: https://github.com/Cargill/splinter/pull/1253


Organize the apps list somehow

The apps list is long and growing (and that's great!) and I'd like a better way to check it. There's already a filter but I think separating them into categories would be more useful sometimes, or maybe it's just me? I mean something like the GitHub explore page

Categories were added: https://www.electronjs.org/apps


In the "PostGIS - using sqlpage with geographic data" example, no points with latitude 0 are shown.

Introduction
When adding points to the "PostGIS - using sqlpage with geographic data" example, when adding a point with latitude 0 it is ignored.
Points with any other latitude are shown.
Example fix at the end of the report.
To Reproduce

Run the example code.
Add a point with lat/long 0 0,

Actual behavior
The point will not show up on the map.
Expected behavior
The point with lat/long 0 0 shows up on the map
Version information

OS: Windows
Database Postgres
SQLPage Version: 0.24.1

Additional context
The map.handlebars template has a bug which filters out latitudes of 0. A fix is in the fork here:
https://github.com/matthieupeeters/SQLpage.bugfixes
PR here:
https://github.com/lovasoa/SQLpage/pull/463

Thanks for both reporting and fixing this !
https://github.com/lovasoa/SQLpage/pull/463


Support the device code flow

Is your feature request related to a problem? Please describe.
When creating CLI or native apps the device code flow is the preferred grant type to get access tokens into a users environment.
Describe the solution you'd like
Examples and logic for supporting the device code flow in this library would be amazing.  I would be happy to contribute.
Describe alternatives you've considered
The only golang library I've seen implement the device code flow on github is solid.  It's a fairly basic flow and shouldn't be too difficult to implement.

Whoops!  Didn't see the existing issue #141 .  Consider this a +1, closing for now.


security concerns of nix store paths in grub.cfg

Issue description
Even if LUKS encrypted volumes is used, unencrypted /boot/grub/grub.cfg has nix store paths:
menuentry "NixOS - Default" {
search --set=drive1 --fs-uuid 11111111-2222-3333-4444-555555555555
  linux ($drive1)//kernels/11111111111111111111111111111111-linux-4.9.130-bzImage systemConfig=/nix/store/33333333333333333333333333333333-nixos-system-x220-19.03.1699 init=/nix/store/33333333333333333333333333333333-nixos-system-x220aj-19.03.1699 boot.shell_on_fail noibrs noibpb nopti noretpoline nospectre_v2 loglevel=4
  initrd ($drive1)//kernels/22222222222222222222222222222222-initrd-initrd
}

Knowing the hash of nix closure<PHONE_NUMBER>3333333333333333333333 in this example) allows the attacker to query nix caches (http-servers like cache.nixos.org or Cachix; private ones can be found using Shodan.io) and download the whole closure, which might contain sensitive information such as credentials to other resources (database servers, etc), SSL certificates, ...
This makes the hash of the system closure as well sensitive and it should not be left in the unencrypted area.

Sounds like a "security by obscurity" problem

I am only about using
"init=/nix/var/nix/profiles/system-12-link/init"
instead of
"/nix/store/33333333333333333333333333333333-nixos-system-x220aj-19.03.1699"
(and the same for systemClosure, kernel and initrd)

The only sensitive information that should be in the system closure is the versions of the packages (well… once encryption will finally be merged). Sure that's enough to help mount attacks, but it's not really bad IMO, especially as most programs are fingerprint-able anyway.
Still, it does make sense to use the anonymized paths… if it's actually possible to rely on them never being reused, which I'm not sure of.

Even then a user can put sensitive information into the nix store by accident, so it seems an easy change to prevent an attacker to exploit it.

That change won't prevent an attacker from exploiting anything. If your secrets are available on a cache server accessible from the internet (which is the assumption for an attack based on that), then you already have bigger problems. Like, most likely timing attacks would allow the attacker to list the contents of your cache anyway without having physical access to your computer (if there isn't an even easier way of accessing it, I haven't checked how binary caches were served).
That said, yes, why not anonymize the paths, it's still a tiny bit better for some marginal use cases, maybe :) (though I haven't seen proof of that yet)
But I think if it means there's a risk of a grub entry being re-used and pointing to the wrong entry, then this idea should be dropped to the floor, because the actual issue is that the secrets have been put in the store in the first place.

because the actual issue is that the secrets have been put in the store in the first place.

Well, even it there is no explicit secrets on the cache server, there are a lot of information useful for an attacker: all system configuration (e.g. firewall rules, IP addresses of other machines, ...) and all the binaries (versions + possibility to check their exploitability offline)

Yes, that's what I was thinking of in my first message when mentioning “versions of packages.” But the thing is that an attacker can likely already get this information by attacking directly the cache without ever having access to the hard drive of the computer using it, by eg. timing attacks or other attacks on the cache.
Basically, I feel that this issue is indeed a security issue, but doesn't respect xkcd's law for knowing which vulnerabilities deserve being fixed quickly: it's likely easier to just hit on the cache than to access the shutdown hard drive of a computer using it.

Knowing the hash of nix closure<PHONE_NUMBER>3333333333333333333333 in this example) allows the attacker to query nix caches

but how? Are you assuming that system derivation is served from some public place?

but how? Are you assuming that system derivation is served from some public place?

Yes.
If the deployment is not limited to one hosting provider, nix-serve should be served from a public place to allow nix-infecting machines on different hostings.
And there is almost nothing to do to make it less public: it should be available from all Amazon, DigitalOcean, Hetzner, Packet's ... IP addresses where a new server would pop up.
And then curl http://my-cache.org/33333333333333333333333333333333.narinfo is enough to start exploring the system closure

And we could ask @domenkozar on how many system closures there are on Cachix.
I guess a lot

nixos machines should be protected with security (cryptography) instead of obscurity (discoverability of nix hashes).

With the current architecture of Nix cache, they are protected only by discoverability of nix hashes.
So far I have seen no proposal for introducing cryptography to protect Nix cashes which are exposed to public Internet. With better Nix cashes it won't be needed to protect the hashes, but it is about the far distant future

@volth Hmm… that is a good point. Maybe you could open a RFC for having an encrypted binary cache protocol? Now, the only use case I could see is your having to serve your configuration to the whole internet for nix-infect (not sure exactly which nix-infect you're talking about, would you have a link?), as otherwise you could just use a private cache (eg. in a VPN). Maybe it'd be possible to fix nix-infect instead?

nix-infect (not sure exactly which nix-infect you're talking about, would you have a link?),

I refer here to the common method of turning

Debian/CentOS machines
machines booted from system-rescue-cd.iso
machines booted from hoster's PXE rescue system

into NixOS by downloading .nars from a public Nix cache and then either kexec new kernel or installing grub and reboot. I have own code but there are plenty open source solution on Github, even nixpkgs has one embedded (grep -i "lustrate") and nixops has another one.

private cache (eg. in a VPN)

That means one have to set up the VPN on zoo of different systems, some of them (PXE booted) do not even have a package manager; and it also implies creation of a non-NixOS system for managing and distributing secrets (VPN keys/passwords).
Perhaps, one-time passwords would help here.

Now, the only use case I could see is your having to serve your configuration

Another is Cachix which presumable containing other's secrets protected only by discoverability of their hashes

I am in the process of implementing authenticated binary caches for cachix, as per https://github.com/cachix/cachix/issues/24 - using https and netrc (basic auth). It's not perfect, but it does the job.

@volth Oh, if you control your system then you could maybe download from<EMAIL_ADDRESS>This would avoid the overhead of setting a VPN and should be relatively secure (unless you assume the initial OS is compromised specially to attack you… in which case it could just edit your closure anyway)
@domenkozar Hmm… I guess this would be a paid feature? At least that sounds like a good way to make cachix self-sustainable by making private binary caches paid, as private binary caches don't profit the overall community :)

@volth Then again, if it's possible to hide the path leaked from grub without drawbacks like nixos-rebuild boot not working in certain circumstances (eg. multiple differently-named generation channels) or entries in grub being re-associated to wrong system generations after a garbage-collection (the two most obvious potential failure modes I just came up with), then why not? I just don't think it should be prioritized over… almost anything else. Now, I'll see myself out, having said all I wanted to say :)

@Ekleog yes, it will be a paid feature of cachix.

and should be relatively secure (unless you assume the initial OS is compromised specially to attack you…

The attack vector I consider here is leaking out the backup images of virtual machines made by hostings (on some hostings it is not even possible to turn it off, they make backup images daily or weekly).
That's why I (and many others, as the feature is embedded in nixops) encrypt the disks of cloud instances with LUKS.
It does not help from advanced attacks, but it protects offline disk images to some extent.
Stealing laptops is another reason why people use LUKS. At least we have to be aware of the hashes left unencrypted in grub.cfg (and some less sensitive hashes in initrd) and what the attacker could recover knowing the hashes.

I think I'm missing something here, so if you don't mind clarifying: the concern is that an attacker would be able to obtain sensitive information from a public cache server, given physical access to an encrypted system (and thus the hash of the systemConfig, since presumably the kernel/initrd aren't particularly interesting)?  In this case, we assume:

The disk is encrypted, so an attacker can't simply read the Nix store
The legitimate user of the system has pushed their entire Nix closure to a cache (since presumably if you don't push the root systemConfig, then you can't fetch that to traverse to anything sensitive).
The cache is public, and an attacker can find it / fetch arbitrary things from it.

Is that an accurate summary?
And if so: given the lack of Secure Boot support in NixOS (#42127, #39286), my general view on this is "an attacker could also just backdoor your boot loader to capture your password and exfiltrate it", so I don't know that this is a huge risk?  Again, possible that I'm missing something - clarification is appreciated!

my general view on this is "an attacker could also just backdoor your boot loader to capture your password and exfiltrate it"

I'd say it is another treat: when your machine had been in the hands in the attacker and then returned to you.
I'm talking only about the attacker obtained your laptop (or cloud disk images) with the disk encrypted. The attacker cannot decrypt it (by installing an backdoor and prompting you to enter the password as though you are unaware of the laptop absence or by using the force or somehow else) and have to rely only on what can be found on the disk.
And there are hashes of systemConfig, which act like access tokens to public caches or private caches exposed to Internet.

No one has come up with a PR, so I think it is reasonable to close this issue. Discussion should continue on Discourse if necessary.


Add Load Tests CoGBK Dataflow Streaming Java github action

The following jenkins job has been migrated:
https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_Streaming
https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_Streaming_PR
How has it been tested?
The job has been tested on Self-hosted GKE runners ubuntu 20.04.
The following test cases are passed:

by adding a comment to the PR

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Will merge once conflicts are resolved


ColumnMenus: if open second menu without closing first, get duplicate items

Open the first menu, then open the second (or last) without closing the first.
http://plnkr.co/edit/mtEO8fkK6ohqQDpgbyGE?p=preview

Tracked to #2845, so have reverted that.  But that functionality is wanted, so we need to fix and then re-merge.

Refer #3049, which is the PR for this.  Waiting on some minor documentation updates then it can be merged.

Closing, track via PR.


Add client assertion authentication

[x] Token endpoint
[ ] Introspection endpoint
[ ] Revocation endpoint

Related to #61

Thanks!
couple of comments (and that's why it is always better to start a discussion first)

I don't want a dependency on the Microsoft JWT lib
I'd rather have a more general way of doing that - e.g. an easy way to ammend the form body

(and you broke the build ;)
So maybe start over and do a proposal first.

Maybe some generic callback when creating the POST body would be a better solution.

You probably want to start over. Agreed?

At least, we some code to discuss :)

I don't want a dependency on the Microsoft JWT lib
Another lib?
If not lib, the hard part will be the signing algorithm probably limited to HS256 and RS256.

I'd rather have a more general way of doing that - e.g. an easy way to ammend the form body
Like here ? https://github.com/IdentityModel/IdentityModel2/blob/dev/src/IdentityModel/Client/TokenClientExtensions.cs#L251

I will open an issue dedicated for this discussion.

I will open an issue so we can discuss this.


Docs: Feature slack more prominently

Resolves #227

Pull Request Test Coverage Report for Build 1072

0 of 0   changed or added relevant lines in 0 files are covered.
118 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-2.1%) to 51.013%

Files with Coverage Reduction
New Missed Lines
%

pkg/rtcp/reader.go
18
81.82%

pkg/rtp/packet.go
44
44.44%

pkg/rtcp/reception_report.go
56
100.0%

Totals

Change from base Build 1046:
-2.1%

Covered Lines:
3323

Relevant Lines:
6514

💛  - Coveralls

Pull Request Test Coverage Report for Build 1072

0 of 0   changed or added relevant lines in 0 files are covered.
118 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-2.1%) to 51.013%

Files with Coverage Reduction
New Missed Lines
%

pkg/rtcp/reader.go
18
81.82%

pkg/rtp/packet.go
44
44.44%

pkg/rtcp/reception_report.go
56
100.0%

Totals

Change from base Build 1046:
-2.1%

Covered Lines:
3323

Relevant Lines:
6514

💛  - Coveralls


Vendorize packaging directly to avoid #909

Vendorize latest dependencies
Release 0.13.12
fixes #909

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

13 of 13 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.7%) to 83.296%

Totals

Change from base Build<PHONE_NUMBER>:
0.7%

Covered Lines:
5406

Relevant Lines:
6396

💛  - Coveralls


Orion -> Tessera migration fails

#Description
To be able to use Tessera it is necessary to migrate the data that we currently have in Orion. Therefore by running the migration component you should be able to migrate the data from Orion to Tessera.
#Steps to Reproduce (Bug)
Follow the steps in the following manual https://docs.orion.consensys.net/en/latest/Tutorials/Migrating-from-Orion-to-Tessera/
Step 3 fail when I execute:

migrate -f "/orion/orion.conf" -o="tessera-migrated.conf" tessera.jdbc.password "Password" tessera.jdbc.url "jdbc:h2:tessera1" tessera.jdbc.user "user1"

Expected behavior: Orion data should be migrated to Tessera
#Actual behavior:
cannot retrieve data from levelDB
return the follow error:
[INFO ] [main] 05-11-2021 17:29:25 n.c.t.m.d.InboundDbHelper - Opening leveldb /root/lacchain/orion/routerdb
Found MigrationInfo{rowCount=0, transactionCount=0, privacyGroupCount=0, queryPrivacyGroupCount=0} to migrate.
java.lang.IllegalStateException: No data found for LevelDB{path = /root/lacchain/orion/routerdb}. Check orion storage config string and/or storage env
at net.consensys.tessera.migration.MigrateCommand.call(MigrateCommand.java:67)
at net.consensys.tessera.migration.MigrateCommand.call(MigrateCommand.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at net.consensys.tessera.migration.Main.main(Main.java:22)
#Frequency: All the time
#Versions (Add all that apply)
Software version: besu/v20.10.2/linux-x86_64/oracle-java-11   besu/1.5.3/linux-x86_64/openjdk-java-11
                             orion/v1.6.0/linux-x86_64/oracle-java-11
Java version: java 11.0.7 2020-04-14 LTS
OS Name & Version: Ubuntu 18.04
Cloud VM, type, size: Local VM 2 core, 8 GB RAM

The error indicates No data found for LevelDB{path = /root/lacchain/orion/routerdb. Could you please double check if this db location is correct?
If it is, please provide your orion config file and some sample data for us to reproduce

Yes, the db location is correct. This foder was created when I started Orion the first time.
Orion Configuration File:
privacy-marker-transaction-signing-key-file="/root/lacchain/data/key"
nodeurl = "http://<IP_ADDRESS>:4040/"
nodeport = 4040
nodenetworkinterface = "<IP_ADDRESS>"
clientnetworkinterface = "<IP_ADDRESS>"
clienturl = "http://<IP_ADDRESS>:4444/"
clientport = 4444
publickeys = ["/root/lacchain/orion/keystore/nodeKey.pub"]
privatekeys = ["/root/lacchain/orion/keystore/nodeKey.key"]
passwords = "/root/lacchain/tmp/.account_pass"
othernodes = ["http://<IP_ADDRESS>:4040"]
#tls = "strict"
tls = "off"
tlsservercert = "/root/lacchain/orion/certificates/orion_cer.pem"
tlsserverchain = ["/root/lacchain/orion/certificates/orion_ca.pem","/root/lacchain/orion/certificates/CAs/orion_ca_col.pem","/root/lacchain/orion/certificates/CAs/cadena-everis-lac-eth-orion-1"]
tlsserverkey = "/root/lacchain/orion/certificates/orion_cer.key"
routerdb.zip

I've looked at the zip you sent and there are no data in there. Also no .sst files present in the folder. Please check if this is the correct directory that you're intending to migrate

The storage value was not set in the config and most of them are using default values.. what use case are you running using the default values assigned by Orion. Please reopen issue with more details


New release

Master branch is ahead of tag v2.0.0 by 19 commits. Can you update the master branch with a newer tag please?

I think this is asking for a new release. See https://voxpupuli.org/docs/releasing_version/ for the release process.

@bastelfreak Can we pull the trigger?

The EL9 fixes from Feb 2022 wills soon become important to me...

I just pushed a new release

and just as a note: Everybody can create a release PR: https://voxpupuli.org/docs/releasing_version/


Organization API - Internal server error when updating organization info

API : PUT organizations/{globalid}

Seems like saving an entire organization in db is not implemented yet
https://github.com/itsyouonline/identityserver/blob/master/db/organization/db.go#L665

Remove this api call and update documentation


Always canonicalize quantity values

IMPORTANT: All PRs must be linked to an issue (except for extremely trivial and straightforward changes).
Fixes #733
Description
Always canonicalize quantity values in the quantity index table.
Alternative(s) considered
Have you considered any alternatives? And if so, why have you chosen the approach in this PR?
Type
Choose one: Code health
Screenshots (if applicable)
Checklist

[x] I have read and acknowledged the Code of conduct
[x] I have read How to Contribute
[x] I have read the Developer's guide
[x] I have signed the Google Individual CLA, or I am covered by my company's Corporate CLA
[x] I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach
[x] I have run ./gradlew spotlessApply and ./gradlew spotlessCheck to check my code follows the style guide of this project
[x] I have run ./gradlew check and ./gradlew connectedCheck to test my changes locally
[x] I have built and run the reference app(s) to verify my change fixes the issue and/or does not break the reference app(s)

@jingtang10 , considering that the reviewer has approved these changes, can this PR be merged?


Add terminal-debug colors

I am using the integrated terminal-debug neovim plugin and I wanted to have nicer colors than the default ones. I thought I'd add an upstream PR for this:

☝ 2 is the breakpoint number here

If you have suggestions for better colors, I'm not married to those ones :)

Looks good, thanks!


Project Euler 004

euler004

Approach:

Basic

iter 1 :- TLE 2 test cases

iter2:- same as (1)


SeatAndEat : Make Hero-section Responsive

Make the hero-section responsive
Find the required files in the repo. Don't make changes to any other css, html file

I would like to make hero section responsive @Rupam-Matabber .

Assigned to you @Pransav290302 . All the best.

Updates @Pransav290302 ?


add index on request table

... so I've just screwed up a pull request...  ._.
One commit to review, the patch file. The other commit is in the other PR.

Current coverage is 20.41% (diff: 100%)

Merging #279 into master will not change coverage

@@             master       #279   diff @@
==========================================
  Files             9          9
  Lines           583        583
  Methods          79         79
  Messages          0          0
  Branches          0          0
==========================================
  Hits            119        119
  Misses          464        464
  Partials          0          0

Powered by Codecov. Last update 042252a...3062e6b


SWC has ghost section if "section_order_break_swc"

This branch: https://github.com/mgeplf/NeuroM/tree/swc_ghost_sections
has a test (neurom/io/tests/test_utils.py:test_load_neuron_section_order_break_swc) that fails: an extra, empty, section is created

I believe it has to do with how neurom/io/datawrapper.py:_extract_sections() is merging gaps:
    if sec.pid in gap_sections:
        _merge_sections(sections[sec.pid], sec)

This leaves the old section around, even though it can't be pointed to by anything else (I think.)  Need to investigate this further when I have time.


Preserve options when templating

There are circumstances where I would like to preserve template variables after processing a link. The primary use case would be for displaying uri routes with partially filled options, yet displaying the totality of the options to the end user.
For example.
If I have a template that looks like
/events{?product,date,days,seats,before,at,after}
I may want to "prefill" the product value and provide the end user a link that looks like
/events?product=10{&date,days,seats,before,at,after}
Currently this is not an option, but it would be helpful in my circumstance if it were.
If this is something you are interested in, I can put in a PR at some point.

This is a great idea!
I can definitely look at a PR for this if you'd like to submit one. That being said, the state of the code isn't great, though the unit test suite is pretty robust.

I'm cool with closing this, however, could we craft a new release? Thanks so much for merging!

Sorry that took so long, but 1.1.0 is released. Thanks for the contribution!

:+1: Thanks so much!


I trusted this lib

I trusted this lib and now my computer is a pile of ash and melted plastic.  You better delete this or I'll send my highly-skilled lawyers after you. And they will find you.  Good day, sir!


No way to specify parameters in @EnableExtendedRepositories annotation

Since @EnableExtendedRepositories now replaces @EnableJpaRepositories, there is no way to specify parameters originally found in EnableJpaRepositories annotation.
@EnableExtendedRepositories should contain parameters like:

basePackages
repositoryBaseClass
transactionManagerRef
entityManagerFactoryRef
and others.

Or provide any other way of achieving the same goal :)

With 1.0.1 you can use can use @EnableJpaRepositories(repositoryBaseClass = SimpleExtendedQueryDslJpaRepository.class, ... other options)

Hi @lpandzic
With the Spring data upgrade it is no longer possible to use that class because it is not public.
What can be done?

With 2.0.0 you should be able to use @EnableExtendedRepositories like mentioned in README.md.

@jcheros , do you need parameters of EnableJpaRepositories?

Yes. I need this
@EnableJpaRepositories (basePackages = "com.socamapa.jpa", repositoryBaseClass = SimpleExtendedQueryDslJpaRepository.class)
But now that class is no longer public and it is not possible to use it with @EnableJpaRepositories

Fixed (https://github.com/infobip/infobip-spring-data-jpa-querydsl/issues/4)


Confusing behaviour of df.empty

This is as much a documentation issue as anything else. Basically it seems confusing that df.empty != df.dropna().empty I.e. that a a dataframe consiting entirely of na is not treated as empty. Obviously this is a bit of an edge case, but it caused a bunch of failures for me when used eith pd.read_sql methods, as database tables will often have columns that are not available for partiicular entities, and so can return an entire series of na.
It seems to me that in all cases df.empty should be the same as df.dropna().empty, but I understand that opinions might differ on this point, but at least the behaviour should be clearly documented.

Can you show an example, the following is works. I agree certainly could update the documentation (with some examples and such)
In [8]: df = DataFrame({'A' : [np.nan]})

In [9]: df.empty
Out[9]: False

In [10]: df.dropna().empty
Out[10]: True

That is exactly the behavior I was questioning, I think out[9] should be True. It seems to me that a dataframe containing nothing by na cells is "empty" according to most definitions.....
Phil

I certainly expected that df.empty would be true if a data frame contained nothing but na cells.

no, that is not a normal definition of empty which is 0-len. nulls are real values which are placeholders. The key here is that you actually have a valid index. Changing this would involve the definition dependent on the data itself which is not a good thing. welcome to have a doc update with some examples though.

Well its certainly not the case that df.empty is the same as len(df.index==0) e.g.
df = pd.DataFrame([], index=[0,1,2])
print df.empty #True
print len(df.index==0) #False

Also
df = pd.DataFrame([], index=[0,1,2], columns=['A','B'])
df.empty #False

So not only do I dispute that len=0 is the semantic definition of empty, that doesn't appear to be the implementation anyway.

Also stuff like:
df = pd.DataFrame([], index=[0,1,2], columns=['A','B'])
df.set_value(1, 'A', 17)
df['B'].empty # False

which just seems plain wrong. If I have a column that I have never added any data to, it should not return false when asked if its empty. I guess under the hood when you specify an index and columns it autofills the dataframe somehow, and that results in this behaviour, but the definition of empty should really play nicely with the default dataframe constructor in these examples imo. Else its just confusing.

its very simple. its empty only if all axes are len 0

It sounds like you're looking for some other collection of Series. A DataFrame is a tabular collection, and it makes sense to look at the shape.

@phil20686 You can eg use:
In [15]: df['B'].isnull().all()
Out[15]: True

@jrebeck my example shows that that is not the implemented behavior:
df = pd.DataFrame([], index=[0,1,2])
print df.empty #True
sum(df.shape) == 0 #false
df = pd.DataFrame([], columns=['A','B'])
print df.empty # True
sum(df.shape) == 0 #false

I really find it super weird that pre-allocation should result in empty=True, e.g. if you concatenate an empty series with a non empty dataframe it will get preallocated and then extracting it means empty has changed from True to False. This seems very strange to me, in some abstract sense series C is the same object, but merely moving it around has changed its properties.
df = pd.DataFrame([], index=[0,1,2], columns=['A','B'])
df.set_value(1, 'A', 17)
series = pd.Series(name="C")
print series.empty
df2 = pd.concat([df,series], axis=1)
print df2["C"].empty

Anyway, my main point is that this behavior should be documented, because its quite counter-intuitive, not to argue about definitions of empty.

@phil20686 every one of those results is correct, what exactly is counter intuitive here?
there isn't any 'pre-allocation' at all. You have indices. If the indicies are 0 in any way (could be 1 dim or not) then you are empty, otherwise you are not.
What exactly are you using .empty? To be honest I have rarely needed this. Most operations in pandas just work regardless if things are empty or not. I think the docs are fairly clear on this.

Um. I had a series that had .empty = True, I concat it with a dataframe, and then I extract the series, and then magically .empty=False? Even though at no time has the user added data to it?
Similarly, you can create a dataframe with either an index or columns but no values and its "empty", but if it has both and no values its "non-empty".
You don't think that is counter intuitive behavior?
Anyway, I think most people would assume that empty == contains no data. That clearly isn't the case as it looks like its the same as
not any(df.shape)

Anyway, my main point was that the documentation of Dataframe.empty should note these behaviors.

@phil20686 one of the highlites of pandas is that it aligns data. When you put in a series it was empty, however, the concat realigned the Series to the other values in the DataFrame
In [19]: df2
Out[19]:
     A    B   C
0  NaN  NaN NaN
1   17  NaN NaN
2  NaN  NaN NaN

In [20]: df2.columns
Out[20]: Index([u'A', u'B', u'C'], dtype='object')

In [21]: df2['C']
Out[21]:
0   NaN
1   NaN
2   NaN
Name: C, dtype: float64

then [21] is clearly NOT empty; yes it is all null. Which is a MUCH more common operation. .empty is a very blunt instrument and not really used much in practice for this very reason. It is correct as far as it goes.

@phil20686 In trying to clear some things up, I think we have to make a distinction between two points:

The definition of empty is clear**: it returns True if the length of one or all of the axes is 0 (you could see it as "len(index) x len(columns) == 0" for a dataframe). It is by defintion not about having all NaNs or not. It is quite possible you find this not the best definition, but taking this definition as a starting point, all the return values in the examples you showed are consistent and as expected.

What is maybe more surprising in some cases, leading to the unintuitive behaviour you described, is the way pandas fills Series/DataFrames with NaNs (and once filled with NaNs, it is not empty anymore). Pandas will fill a DataFrame with NaNs once it has both an index and columns. And there are indeed operation (eg concat) where index/columns can be added, leading to filling with NaNs.

I just want to point out that I think the confusion you get has another root cause than the empty method.
And the current empty method is just not the method you are looking for I think. It would be more something like allnan or allnull, which you can obtain with isnull().all()
** I don't say it is 'clear' in the docs, I mean in implementation
But indeed, the docs of empty can certainly point that out. Do you want to do a PR to specify that this is not about NaNs ?

I agree that the docs of empty could point this out (I actually had a student just ask me about this). Since it's been a few days, I can do a quick PR.


Components wishlist

First components that I can think of:

[ ] Card
[ ] Panel
[ ] Pagination
[ ] Dropdown
[ ] Message
[ ] Collapsible Card
[ ] Date Picker
[ ] Time Picker

If you are working on any of these, please leave a comment and I will edit this list. If I missed something, let me know!

I can work on the cards component

I have most of the pagination component complete. I have a busy week this week but hopefully on the weekend I can finish it off and submit it for review.


icon not show? how to set from with proper font-family to make it work?

how to specify icon in different font-family?
waiting for your docs ....
thanks.

you can either use react-native-pure-icon or react-native-vector-icons


Remove ELEM shortcode option for import process

Why?

It adds complexity that makes import.thor harder to read, but is rarely if ever used.

In my next couple of PRs, I'll be changing import.thor to clear a path for importing New Bedford data and this will be a stumbling block. The ELEM code creates another set of configuration for each new district we add.
I can't remember the last time I used the ELEM shortcut for Somerville, if ever.
It doesn't look like it would even work: the school_local_ids method that calls is is only used in #validate_schools, but not in the code that runs the import itself.

QA'ed this locally under the 3 main conditions I could think of:

No --school argument passed
--school argument passed with one school
--school argument passed with multiple schools

Responded correctly under all three conditions! 👌
Going to selfie this one since it's changing an option that's not currently in use.


Support for @Many2Many multiple assignment

Hi there,
as far is i see there is only limited support for connecting objects of different type:

add(Model child)
remove(Model child)
setParent(s)  -> only for one to many relationships

Couldn't the framework be extended in order to better support

many to many relationships
multiple assignments (e.g. physician.patients = patients), which removes not needed and sets
the named relationships ?

Thank you for considering this.

@rivella50, the framework takes care of three types of relationships:

one to many http://javalite.io/one_to_many_associations
many to many http://javalite.io/many_to_many_associations
polymorphic associations: http://javalite.io/polymorphic_associations

Is this helping?

@ipolevoy i was more asking for an extension of the framework, not an overview of
the existing relationship types.
Mainly a setter method which allows creating rows in the join table with one call, i.e.
the has_many :through association:
http://edgeguides.rubyonrails.org/association_basics.html#the-has-many-through-association

@rivella50, understand what you want better now. Basically you want:

Doctor doctor = ...;

List<Patient> patients = ...
doctor.add(patients);

where patients can be a set of new models instances (do not yet exist in DB), or existing (have proper IDs set)
Is this something you think framework is lacking?

Since i create also Ruby on Rails projects i would say that i miss such a feature here, yes.
Here i just meant updating the collection of join models with one call, i.e. the associated Patients already exist in database.
As a sidenote: When you would need to create a new doctor's patient you would write something like
doctor.patients.create(params[:patient])

Therefore the framework would need to

determine the correct join table
delete rows from that join table which are not in the list of just assigned models
insert rows in the join table for not yet found associations that are in the list of just assigned models

@rivella50 , ActiveJDBC is a close to ActiveRecord, but not a clone.
so you want this:

determine the correct join table

this works automatically

delete rows from that join table which are not in the list of just assigned models

I'm not sure this is expected. So when I see this:
doctor.add(patients);

why would it delete other patients? IMHO, this only adds new ones to the doctor. Also, if you delete join records only, you are risking to have orphan patients left in your database.

insert rows in the join table for not yet found associations that are in the list of just assigned models

This is already working with:
doctor.add(patient)

The only difference, it operates on one child at the time, not a collection.

That's why i proposed a setter method, which does that (deleting not needed anymore join rows,
inserting new join rows, keeping existing and still needed join rows) all in the background of
the framework :-)
doctor.add(patients);
doctor.remove(patients);

Would be other useful additions.

@rivella50 not a bad idea. However, most methods for processing relationships are mirrored in all relationships. For instance:
doctor.add(patient);

looks the same for many to many, one to many and even for polymorphic associations. This means that new methods like:
doctor.add(patients);
doctor.remove(patients);
doctor.set(patients);

should also be implemented for: many to many, one to many, and polymorphic. In other words, this will require more implementation that you might think :)
However, having these capabilities will make the framework better. By the way, are you also using ActiveWeb?
making sense?

Yes, these enhancements would make sense for all relationship types.
I would guess that this needs some time to implement, definitely :-)
At the moment i only use ActiveJDBC in a Java Swing application.
But if the need for a Java Web application is there in another project i will give ActiveWeb a try.

@rivella50 yes, I think this is a 4 - 5 full time day project to add these features. Will also involve generics:).
I currently have no bandwidth to dedicate this time. If you have some spare time, you can send a pull request and I will review/incorporate changes.

this will include: tests, code for all relationship types, tests passing on all databases, JavaDoc and finally documentation for the site.

Uuuuuhhhhhh, challenging.
I'll see what i can do.

@rivella50 , I'm closing this as a stale issue.
Technically speaking, you can implement this as a single line in Java 8:
patients.foreach( doctor::add);


impl(pubsub): add events to batch callback api

Part of https://github.com/googleapis/google-cloud-cpp/issues/13720

This change is

PR is ready for review


fakeloader doesn't work with 3x version of jquery

Don't use the slim 3.3.1 version of jquery for fakeloaderjs because in 3x version of jquery timeToHide does not work.

It works fine for 3.3.1 version of jquery not with slim 3.3.1 version of  jquery. If using it with bootstrap4 remove the slim version of jquery for bootstrap.min.js

It still doesnt work.

i changed to non-slim jquery version.