Sf expansion 481

Pull Request submission
Insert detailed bullet points about your changes here!
Instructions:

Run the pipeline on the sf_expansion branch
View the output in the R:\BERD Results System Development 2023\DAP_emulation\imputation\imputation_qa folder
filter the data on short forms (formtype==6)
Filter to just one imputation class (column "imp_class"). We suggest class "C_AH", but you could try others
You should see "imputation_***" columns, where *** is the breakdown variable (such as "204" or "303"). Check the variables in these columns exist
For any record you can check the results using Excel formulas.

*Let the reviewer know what data files are needed (and if applicable, where they are to be found)
Closes or fixes

Detail the ticket(s) you are closing with this PR
Closes #481

Code

[ ] Code runs The code runs on my machine and/or CDSW
[ ] Conflicts resolved There are no conflicts (I have performed a rebase if necessary)
[ ] Requirements My/our code functions according to the requirements of the ticket
[ ] Dependencies I have updated the environment yaml so it includes any new libraries I have used
[ ] Configuration file updated any high level parameters that the user may interact with have been put into the config file (and imported to the script)
[ ] Clean Code

[ ] Code is as PEP 8 compliant as I can humanly make it
[ ] Code passess flake8 linting check
[ ] Code adheres to DRY

[ ] Type hints All new functions have type hints

Documentation
Any new code includes all the following forms of documentation:

[ ] Function Documentation Docstrings within the function(s')/methods have been created

[ ] Includes Args and returns for all major functions
[ ] The docstring details data types

[ ] Updated Documentation: User and/or developer working doc has been updated

Data

[ ] All data needed to run this script is available in Dev/Test
[ ] All data is excluded from this pull request
[ ] Secrets checker pre-commit passes

Testing

[ ] Unit tests Unit tests have been created and are passing or a new ticket to create tests has been created

Peer Review Section

[ ] All requirements install from (updated) environment.yaml
[ ] Documentation has been created and is clear - check the working document
[ ] Doctrings (Google format) have been created and accurately describe the function's functionality
[ ] Unit tests pass, or if not present a new ticket to create tests has been created
[ ] Code runs The code runs on reviewer's machine and/or CDSW

Final approval (post-review)
The author has responded to my review and made changes to my satisfaction.

[ ] I recommend merging this request.

Review comments
Insert detailed comments here!
These might include, but not exclusively:

bugs that need fixing (does it work as expected? and does it work with other code
that it is likely to interact with?)
alternative methods (could it be written more efficiently or with more clarity?)
documentation improvements (does the documentation reflect how the code actually works?)
additional tests that should be implemented (do the tests effectively assure that it
works correctly?)
code style improvements (could the code be written more clearly?)
Do the changes represent a change in functionality so the version number should increase? Start a discussion if so.
As a review you can generates the same outputs from running the code

Your suggestions should be tailored to the code that you are reviewing.
Be critical and clear, but not mean. Ask questions and set actions.

Looks great! And the numbers all add up in the outputs (at last!).
Only question from me @AnneONS is do we need to update the imputation_marker column for short form expansion?

I'd say no as ALL short forms undergo sf_expansion, so there's no point. I suggest imp_marker is just where we do TMI, MoR actual imputation (sf expansion is called imputation, but it's rather different I'd say) I'll ask Ana

Looks great! And the numbers all add up in the outputs (at last!).
Only question from me @AnneONS is do we need to update the imputation_marker column for short form expansion?

I'd say no as ALL short forms undergo sf_expansion, so there's no point. I suggest imp_marker is just where we do TMI, MoR actual imputation (sf expansion is called imputation, but it's rather different I'd say) I'll ask Ana

Confirmed with Ana, she agrees.

Looks great! And the numbers all add up in the outputs (at last!).
Only question from me @AnneONS is do we need to update the imputation_marker column for short form expansion?

I'd say no as ALL short forms undergo sf_expansion, so there's no point. I suggest imp_marker is just where we do TMI, MoR actual imputation (sf expansion is called imputation, but it's rather different I'd say) I'll ask Ana

Confirmed with Ana, she agrees.

Makes sense to me!


Figure out nix build

This template provides a devShell environment (nix develop), but it doesn't really provide a Nix package (nix build). We should provide one, so that nix run github:srid/leptos-nix-template will directly run this app (production version) on any machine.
https://github.com/srid/leptos-nix-template/blob/09b5b2da34d9efb496fa052be793abfb5cf609a9/flake.nix#L95-L96
@benwis has provided a starting point here https://github.com/leptos-rs/leptos/issues/1152#issuecomment-1616135687

I think we can solve this by doing #3 followed by switching to https://trunkrs.dev/
Then we can write a derivation like: https://github.com/OliverEvans96/yew-bevy-nix-flake/blob/cbbf60e7ed96010dc8d74f719da95ba08fd561c3/flake.nix#L29-L48

@benwis Trunk + crane works great!
I have something working right now: https://github.com/srid/leptos-fullstack
Need to figure out SSR next (I'm afraid that might force me to go back to cargo-leptos? Unless cargo-leptos works well with Crane .... does it?)

@benwis Trunk + crane works great!
I have something working right now: https://github.com/srid/leptos-fullstack
Need to figure out SSR next (I'm afraid that might force me to go back to cargo-leptos? Unless cargo-leptos works well with Crane .... does it?)

My example at the top of this thread uses cargo-leptos for SSR, you could use cargo and wasm-pack, but I don't have an example of that in Nix


[BUG] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position
Here is my code:
from curl_cffi import requests
resp = requests.get('https://cupoworld.es/tiendas/R')
print(resp.text)

I use requests lib can get correct answer, but use curl-cffi get encoding error.

requests ignores the decode error
content = str(self.content, encoding, errors="replace")

while curl_cffi does not.
The behavior will be changed to the same as requests in the next version. In the meanwhile, you can decode by your self.

Can you please give an example for it, I tried to fix it with decode but without success.

r = requests.get("https://example.com")
text = r.content.decode("utf-8", errors="replace")


RFC: Extend Secure Sockets with engine support

Introduction
Zephyr implements the concept of Secure Sockets. Those ease the integration of TLS and DTLS into arbitrary applications. However, the secure sockets hide the underlaying crypto library. Moreover, the current code lacks support for opaque private keys and engine callbacks.
The secure socket API should be extended with an API that allows replacing any of the asymmetric functions on a per context basis.
Problem description
mbedTLS has support for alternative crypto implementations and limited (RSA: yes, ECP: no) support of opaque keys. However, Zephyr’s code that setups the pk context is very rigid and cannot cope with situations where TLS_CREDENTIAL_PRIVATE_KEY is not set.
Proposed change
Add support for adding opaque private keys with TLS_CREDENTIAL_PRIVATE_KEY_BLOB in addition to the existing TLS_CREDENTIAL_PRIVATE_KEY and allow for the definition of alternative cryptographic functions.
Detailed RFC
Proposed change (Detailed)
The Secure Socket subsystem runs to the best of my knowledge in kernel space. The proposed change therefore needs to take privilege separation into account. User space should not be able to set functions that are executed in kernel mode.
mbedTLS features mbedtls_pk_setup_rsa_alt which allows the definition of mbedtls_pk_rsa_alt_decrypt_func, mbedtls_pk_rsa_alt_sign_func and mbedtls_pk_rsa_alt_key_len_func. Those could be exposed to the developer to be defined at compile time. Additionally, tls_set_own_cert could be extended to detect opaque keys and call mbedtls_pk_setup_rsa_alt instead of mbedtls_pk_setup with the above alt function pointers. Finaly Zephyr should provide a macro/function that allows for a type safe recovery of the KEY_BLOB within the alt-functions from void *ctx. Blow the function typedefs:
typedef int (*mbedtls_pk_rsa_alt_decrypt_func)
  (void *ctx, int mode, size_t *olen, const unsigned char *input,
   unsigned char *output, size_t output_max_len );

typedef int (*mbedtls_pk_rsa_alt_sign_func)
  (void *ctx, int (*f_rng)(void *, unsigned char *, size_t), void *p_rng,
   int mode, mbedtls_md_type_t md_alg, unsigned int hashlen,
   const unsigned char *hash, unsigned char *sig );

typedef size_t (*mbedtls_pk_rsa_alt_key_len_func)
  (void *ctx );

Dependencies
The proposed change addresses RSA only. The three alt functions described above are not yet available for ecp. We would need to create a PR in the mbedTLS project to upstream mbedtls_pk_setup_ecp_alt.
Concerns and Unresolved Questions
Depending on the application an engine might need to communicate with a UI to request a password. The current RFC only addresses one-way data transfers.
Alternatives
I've created a Proof of Concept for using a Trusted Platform Modules (TPM2) as secure storage of private keys. The PoC defines MBEDTLS_ECDSA_SIGN_ALT to catch all sign-requests and forward them to the TPM2. However MBEDTLS_ECDSA_SIGN_ALT was designed with crypto accelerators in mind and does not offer a context, see the function prototype below:
int mbedtls_ecdsa_sign(mbedtls_ecp_group *grp,
                       mbedtls_mpi *r, mbedtls_mpi *s,
                       const mbedtls_mpi *d,
                       const unsigned char *buf,
                       size_t blen,
                       int (*f_rng)(void *, unsigned char *, size_t),
                       void *p_rng)

Therefore I had to pass KEY_BLOB on a side channel. I chose a global variable. Moreover since MBEDTLS_ECDSA_SIGN_ALT lacks the context, this approach only works for a single key per algorithm (1x RSA, 1xECP).

This looks more like a security enhancement than network one to me. cc: @d3zd3z @ceolin

I think this is a good start. Opaque keys will be important when wanting to store keys in a secure environment such as TF-M/PSA.

Hello,
Any news on this feature which could really enhance security ? Would be interested!

At this point, we are waiting for some more progress in how this is done within TF-M. It works when chosen at build time, and there was additional information added to the keys, but it isn't yet possible to have keys that live in different places from how the system was configured.


Remove payment_method_id from accountUpdateSetup

It's not being processed by API.

@kitop is it still true that this field is not doing anything right now?

@kitop is it still true that this field is not doing anything right now?

Yup! Still true

Cool. I'm going to add a Regeneration of the go client to the branch and them merge


Question: How to capture full traceback from resolver exceptions?

When an exception is raised in a resolver, such as KeyError, the result is returned to the client as a TartifletteError/GraphQLError instance.  That's ok for the client but makes development and debugging difficult as the actual exception type is lost.
I know I can implement a global error coercer which could retain the actual type of the exception. However I'm still missing out on the traceback and context variables that I could see with e.g. cgitb, or logging.exception.
More specifically, I would like to capture all exceptions and send them to Sentry.
To do this, I need access to exception context from sys.exc_info()
I've been poking around the code to see if I can find where, e.g. a resolver KeyError would get caught and then converted to TartifletteError.  I even tried subclassing Engine and adding a try/except to execute(), but that didn't work.
At this point my only option is to add another decorator to all my resolvers to catch any exceptions. I'd rather not do that.
Can you please suggest where in the code, if at all possible,  I could catch low-level exception details?
Thanks

ah, so its resolver/factory.py that's doing the catch.. hmm. looks like I'll have to write  a decorator for every resolver.

@bkcsfi I think @Maximilien-R has an easier solution for you here. I'm assigning this issue to him. I let him give the details to you.

Hi @bkcsfi,
In my opinion, for your use case, the easiest way to log exception wrapped into a TartifletteError is to use the error_coercer feature (cf. https://tartiflette.io/docs/api/error-handling#advanced-add-a-global-error-coercer).
You could have something like that:
import logging

from typing import Any, Dict, Union

from tartiflette import Engine

logger = logging.getLogger(__name__)

def logger_error_coercer(
    exception: Union["TartifletteError", Exception], error: Dict[str, Any]
) -> Dict[str, Any]:
    original_error = getattr(exception, "original_error", None)
    if original_error is not None:
        logger.error(original_error, exc_info=original_error)
    return error

engine = Engine(
    "your-sdl.graphql",
    error_coercer=logger_error_coercer,
)

FYI, Tartiflette catch every exceptions which occurs during the resolving process. If an exception isn't coercible (it doesn't implement a coerce_value method) it will be wrap into a GraphQLError but the original exception will be set to the original_error attribute. I think this part isn't documented yet but it should come soon :)
I hope this solution will meet your needs. Otherwise, do not hesitate to come back to me.

The value set to the original_error attribute is the exception that is returned here which occurs during the resolving process. Meaning that it should contains the full original exception information including its traceback.
Regarding the fact that on my proposed code, exc_info parameter is the original exception, its not a problem since according to the documentation, this paremeter can be an exception instance (and in this case, the exception data are used instead of sys.exc_info()):

There are three keyword arguments in kwargs which are inspected: exc_info, stack_info, and extra.

If exc_info does not evaluate as false, it causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) or an exception instance is provided, it is used; otherwise, sys.exc_info() is called to get the exception information.

Also, I just tested my solution on the dogs example from the tartiflette-aiohttp project.  You should have something like:

logger-error-coercer-patch.txt

Thanks so much for your assistance with this issue.
What threw me off on this is that sys.exc_info docs suggest it only works properly if its called from  within an except clause.
However your sample code does work. Below is what I implemented, passing exceptions to sentry, or if that's not configured using logging.exception (I could also use cgitb here to get context vars).
def logger_error_coercer(exception, error=None):
    original_error = getattr(exception, "original_error", None)
    if original_error is not None:
        if not sentry_sdk.capture_exception(original_error):
            logging.exception(
                "resolver error", exc_info=original_error, stack_info=True
            )
    return error

Sentry does in fact capture context variables on the stack as shown below


Styling marquee selection

Is there an easy way to style the marquee selection element? The default is light blue with a dashed border.

You could overwrite the CSS rules.
This is the default style
.react-flow__nodesselection-rect {
  background: rgba(0, 89, 220, 0.08);
  border: 1px dotted rgba(0, 89, 220, 0.8);
}


Caculating the default ttl in milliseconds

This commit fixes the ttl calculation.
Date#getTime() returns the number of milliseconds since 1 January 1970 00:00:00 UTC, therefore this fix the ttl duration window which was being mistakenly calculated in milliseconds. The user passes (and also the default ttl) was in seconds.

@benlangfeld is there anything else missing?

Sorry for the delay @guimello, busy week! This looks great, thank you for your contribution!


Bnn fix

exposes some hyperparameters of Bohamiann and makes the unittests run faster (see also #195 )
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #200 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #200   +/-   ##
=======================================
  Coverage   89.47%   89.47%
=======================================
  Files         105      105
  Lines        2861     2861
  Branches      292      292
=======================================
  Hits         2560     2560
  Misses        249      249
  Partials       52       52

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80852ea...e132c51. Read the comment docs.


pubsub support

why not support pubsub while run in browser?

Hi @mccoysc please see the discussion here: https://github.com/ipfs/js-ipfs-api/issues/518


dts: bindings: dma: correct compatible name of Intel SEDI dma controller

Replace an underscore with a hyphen in the name to align with the general naming convention.

should update release notes and potentially migration guide

should update release notes and potentially migration guide

How does that work, @decsny - @kwd-doodling should update it when the 3.7 release note is created?


DRILL-3497: Throw UserException#validationError for errors when modifying options

Mistakenly closed https://github.com/apache/drill/pull/91. Please review 4c2b698.

Addressed review comments mentioned on #91


synaptics: fix to allow for more scrolling choices

This should be backwards-compatible, though that's assuming that I understand how default values work.

LGTM.


AlexNet_TinyMS_JeffDing

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@JeffDing Thanks for pulling this request, but unfortunately AlexNet has been already merged into the main branch.

@JeffDing 很遗憾告知您跑通AlexNet模型流程的时间在所有参赛选手中位列第4，不满足二等奖的条件（TOP3）

活动已结束，关闭


file not found when browsing the portal

the latest version of superset, when browsing from the browser there are two asset files missing
/static/assets/dist/css-theme.0.15.1.entry.js
/static/assets/dist/common.0.15.1.entry.js

<EMAIL_ADDRESS>is the current version.

Make sure these boxes are checked before submitting your issue - thank you!

[ ] I have checked the superset logs for python stacktraces and included it here as text if any
[ ] I have reproduced the issue with at least the latest released version of superset
[X] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version
0.15.1
Expected results
goto http://localhost:8088/
login
Actual results
there are missing asset files, page renders without css and files mentioned in the first post to the issue
Steps to reproduce
on ubuntu install it via
sudo apt-get install build-essential libssl-dev libffi-dev python-dev python-pip libsasl2-dev libldap2-dev python-numpy

sudo pip install numpy --proxy=$HTTP_PROXY
sudo pip install pandas --proxy=$HTTP_PROXY
sudo pip install cffi --proxy=$HTTP_PROXY
sudo pip install superset --proxy=$HTTP_PROXY

sudo pip install flask-migrate flask-script flask-testing flask-sqlalchemy humanize gunicorn markdown pandas parsedatetime pydruid PyHive python-dateutil requests simplejson six sqlalchemy sqlalchemy-utils sqlparse thrift thrift-sasl werkzeug  --proxy=$HTTP_PROXY --upgrade

bring up the server
# Create an admin user with credentials (admin/admin)
sudo fabmanager create-admin --app superset

# Initialize the database
sudo superset db upgrade

# Load some data to play with
sudo superset load_examples

# Create default roles and permissions
sudo superset init

# Start the web server on port 8088
sudo superset runserver -p 8088

I can just use a older release of the codebase 0.15.0 to see if it still has this issue.

@BardiaAfshin missing static files usually is matter of compiling your own build and and forget to build them. Since it's not your case to rule out a misconfigured proxy just point to 8088 and see if it loads fine.
BTW If you are using sudo to install superset you are doing something wrong, no need to run superset with privileges, expecially if it's proxied.

Notice: this issue has been closed because it has been inactive for 467 days. Feel free to comment and request for this issue to be reopened.


ENYO-3467: Unexpected focus jump if pressing 5way while transitioning

lightPanels
Issue:

LightPanel pause spotlight on preTransition and resume spotlight on
postTransition. And give default focus on active panel in async call.
Spotlight recover focus when pressing 5way key while current focus is
not spottable or when no focus under pointer in pointer mode.
But, 2) can lead unexpected focus jump to first control in screen if press
repeated 5way key while panel transitioning.

Fix:
Focus recover routine works only when current focus is not spottable
(null). So, we give focus to placeholder temporary between call
checkSpottability and setTimeout. We skip updating last focus of panel
when placeholder get focused to make setTimeout can give focus to actual
last focused child but not placeholder.
If app give default focus by itself just after checkSpottability and
before setTimeout works, spotlight will safely move focus to specified
default control. And disableSpotlightPlaceholder will set spotlight
false on placeholder. Then setTimeout will give focus to panel and it
will convey focus to the specified default control again.
The isSpottable return true when panel have spotlight container control.
And this will make placeholder spotlight false. This kind of situation
can happens when panel have scroller but not items in it. Panel will not
set spotlight true on placeholder in this case. The isSpottable have
second parameter that is skipping container control. By adding this, we
can set spotlight true on placeholder.
Enyo-DCO-1.1-Signed-off-by: Kunmyon Choi<EMAIL_ADDRESS>

Closing PR as Jira has been closed.


anchor pillar block hash into the pbft block

Purpose

How does the solution address the problem

Changes made

prnet cleaned-up successfully


Regex search

Implement service to search GitHub repositories by regex and keywords

https://docs.github.com/en/search-github/github-code-search/understanding-github-code-search-syntax


TypeError: forward() takes 3 positional arguments but 4 were given

Hi Ross,
As you suggest, I have try to rerun the code from Alex here https://www.kaggle.com/shonenkov/training-efficientdet.
The notebook runs well using kaggle instance, but it gets error message: "TypeError: forward() takes 3 positional arguments but 4 were given", while I am trying to run on my local machine. My packages are effdet version 0.18.2, Pytorch version 1.5, and timm version 0.1.30.
I have also refer the code to wrap like this
class ExtendDetBenchTrain(DetBenchTrain):
def init(self, model, config):
super(ExtendDetBenchTrain, self).init(model, config)
def forward(self, x, target):
    class_out, box_out = self.model(x)
    cls_targets, box_targets, num_positives = self.anchor_labeler.batch_label_anchors(
    x.shape[0], target['bbox'], target['cls'])
    loss, class_loss, box_loss = self.loss_fn(class_out, box_out, cls_targets, box_targets,      num_positives)
    output = dict(loss=loss, class_loss=class_loss, box_loss=box_loss)
    return output

but it still gets the same error
Can you give me an advice.
Many thank
Linh

The code here has changed from when the notebook was written so very minor adjustments are needed. Two of the target box/cls args need to be combined in a dict. Should be staightforward if looking at both versions.

Thank Ross

On Jun 25, 2020, at 01:00, Ross Wightman<EMAIL_ADDRESS>wrote:
Closed #42 https://github.com/rwightman/efficientdet-pytorch/issues/42.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/rwightman/efficientdet-pytorch/issues/42#event-3478667959, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFKZ27AC7RX67IWLW74F4P3RYI5M3ANCNFSM4OGI763Q.

@rwightman
Hi. Nice to meet you.
I have got the same error as mentioned in this issue
So, I change the forward function like this by following

Two of the target box/cls args need to be combined in a dict.

But I stil got the same error.
Could you tell me what might be wrong?

def forward(self, x):
x = self.backbone(x)
x = self.fpn(x)
x_class = self.class_net(x)
x_box = self.box_net(x)
return {x_class: x_box}

@rwightman
The code here has changed from when the notebook was written so very minor adjustments are needed. Two of the target box/cls args need to be combined in a dict. Should be staightforward if looking at both versions.
Hi.
I have got the same error. End I tried but stil suffer it.
Could you tell me the part of code to be fixed?
Regards,
Ryo

Sorry I thought I found the part!
Thank you.

Sorry, after I used the old version, I could run the training.


WPF Map Control Unresponsive (Mapsui.Wpf-Net472)

I just updated a few users to Mapsui.Wpf-Net472. Users who are running 11th Gen Intel i7 chips are experiencing reproducible non-responsive WPF MapControl.
After navigating to the 'map page', where the users expect to see a map, these users see a white/blank page. After resizing the application by stretching the edges of the application, the screen goes from white/blank to rendering appropriately. Sadly, the map is rendered, but the map is unresponsive/frozen.
Users who aren't running 11th Gen Intel i7 chips aren't experiencing this issue. I can provide more context, but I wanted to start the thread here first, and answer specific questions that anyone may have.

Are you eure that it ist mapsui.wpf- net472. Because as far as I Knie mapsui.wpf only Supports net48.

If it is a special CPU problem, it could be a Skia problem.

Are you sure that it is mapsui.wpf- net472. Because as far as I know mapsui.wpf only Supports net48.

Thank you will swap Skia packages to see if there's any change.

If it is a special CPU problem, it could be a Skia problem.

Thank you. Good idea. I'll swap Skia packages to see if there's a change in behavior.

The Mapsui.Wpf-Net472 package is not created by us. I had not seen it before. It is created by jwikberg

His nuget: https://www.nuget.org/packages/Mapsui.Wpf-Net472/
His fork: https://github.com/jwikberg/Mapsui

I'd be surprised if this is related to targeting .Net 4.7.2 instead of 4.8.
@blwhttngtn Can you check if the official Mapsui package has the same issue on 4.8 with this CPU/GPU?

Is UseGPU=false for MapControl? This has caused caused problems in the past. https://github.com/Mapsui/Mapsui/issues/1191

Wegen this helps it ist probably an Intel Graphocs Driver issue. Because Mapsui uses the Gpu to Render the Maps.

@inforithmics Yes, you're right it's an Intel Graphics Driver Issue! I've been able to move past the issue temporarily by asking our affected users to download an Intel Graphics Driver update. @pauldendulk, I hadn't modified the default value of MapControl's UseGPU.

I'd be surprised if this is related to targeting .Net 4.7.2 instead of 4.8. @blwhttngtn Can you check if the official Mapsui package has the same issue on 4.8 with this CPU/GPU?

You're correct -- targeting was not the issue.

Perhaps we should change the default. I will create an issue.

Assuming this another UseGPU issue.


de.creativecommons.org SSL certificate broken

https://de.creativecommons.org inaccessible because SSL certificate error
When you go to https://de.creativecommons.org, the connection fails because the SSL cert is issued to common name "*.newthinking.de" and not to "de.creativecommons.org".

NET::ERR_CERT_COMMON_NAME_INVALID
Subject: *.newthinking.de
Issuer: thawte DV SSL SHA256 CA
Expires on: 07.12.2018
Current date: 01.02.2017
[...]
You cannot visit  de.creativecommons.org right now because the website uses HSTS. Network errors and attacks are usually temporary, so this page will probably work later.

The site now is not accessible with this browser anymore. Even trying to access http://de.creativecommons.org now redirects to https://de.creativecommons.org which leads to this error. The site is now inaccessible to me.
I'm using Chrome 56 Linux 64-bit and verified the issue also with Firefox 51.0.1 Linux 64-bit
--
Finally, we're going to put someone's name here, so they get alerted to the new issue and can assign it to the right person. @robmyers @little-wow

Thank you for reporting this!
We will forward this information to the CC Deutschland team.

@sleeksorrow CC Deutschland are aware of the problem and are working to move to a new server but it will take them a while to do so.


arm64 build fails

In trying to build for an Raspberry Pi 4 running Ubuntu 20.04LTS 64-bit server based on the instructions failed with the following.
Is it possible to provide a compatible IDE or updated build instructions for Ubuntu 20.04 LTS 64-bit? It appears likely this would work with 64-bit Raspberry Pi OS once it is out of beta.
Thank you.
origin	https://github.com/openmv/openmv-ide.git (fetch)
origin	https://github.com/openmv/openmv-ide.git (push)

git status
On branch master
Your branch is up to date with 'origin/master'.

apt install build-essential libgl1-mesa-dev
apt install qtcreator qt5-default qtdeclarative5-dev libqt5serialport5-dev

Qt Creator 4.11.0
based on Qt 5.12.8 (GCC 9.3.0, 64bit)

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"

uname -a
Linux ubuntu 5.4.0-1013-raspi #13-Ubuntu SMP Mon Jun 15 03:17:37 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

cd openmv-ide
./make.py --rpi &> build.log

build.log

You have to fix the build errors yourself. If you can get it building then please send a PR for the fixes in.

Note, I don't build the IDE on the RPI. I build it on a VM running Ubuntu with a cross's compiler using the qtrpi project.


Autoscaler revamp

This is a slight refactoring and retouching of the autoscaler. This fixes several bugs, some of which are blockers for certain niche use-cases. Notably:

Services weren't being treated specially by the autoscaler. Services are one case where the degree of parallelization really, really matters. If you have multiple services, they may all need to be running simultaneously before any real work can be done. Previously, if your services had a fairly short runtime, the autoscaler would believe that it could pack them serially, one after the other. This would lead to service deadlocks in some situations.
New node reservations (reservations are sort of like "time-slices" of resource usage at a node, so you can find the projected resource usage at each point in time) weren't split, meaning that a 5-minute job that uses 5 cores would be projected to use 5 cores for the entire 1-hour "wallTime" of the instance. This means in certain situations, more nodes would be created than would be optimal.
A long-running job, which "extends" a node reservation for a very long time, could end up causing other jobs to be packed into its extended reservations, even though they may be projected to start several hours in the future. This ends up underprovisioning nodes if you have a few long-running jobs and many more short jobs. (I've noticed the old pre-multiple-node-types scaler underestimating the number of needed nodes with Cactus, though I don't know whether that's caused by this particular bug or not.)
The ignored nodes functionality didn't work at all, or when it did, only rarely, because the ignoredNodes set was based on IP addresses, and the Mesos batch system was checking hostnames to see if they were ignored.
When running services or other long-running jobs, the ignored nodes functionality would rarely lead to deadlocks, where all N nodes were ignored despite wanting to scale up to N nodes. Now the scaler will "unignore" nodes if it changes its mind about scaling up or down.

For context, the original scaler worked like this: take the resource requirements of the last 10000 completed jobs, attempt to figure out how many nodes would be able to run those so that all jobs started within an hour. Then multiply that by the length of the current queue times a constant "alpha packing" factor. This is nice from a performance point of view (because it only needs to pack at most 10000 jobs), but leads to some really weird results, because it used the past to predict the future. It also isn't ideal when you absolutely need parallelism (services), because you never want to under-provision like you would with an alphaPacking < 1.0. The current scaler is a blend of using "recent" jobs vs. queued jobs, with alphaPacking determining the mix between the two.
This alters the scaler so that it works entirely on the basis of queued jobs, and the "alpha" parameter is a time. The packing attempts to spawn enough nodes that all the queued jobs will be started by that time. Intuitively this is just a parameter which controls the degree of parallelization, without (hopefully) causing deadlocks if it can help it.
Anyway, this has been mangled a bit since I last tested it thoroughly, because it's been through a rebase and now selectively cherry-picked off my fork. So it's not ready to merge just yet, but it should be relatively OK.
Some considerations:

The scaler basically attempts to set the rate of job completion (assuming homogeneous jobs for simplicity) to n/a, n being the number of jobs and a being the alpha factor. But the scaler will run again in (by default) 60 seconds, and assuming some jobs have been completed, it will try to set the rate of job completion lower (because n is lower). The total number of jobs remaining, n, will decrease from the starting point N as roughly n = N/(t/a + 1). So the end result isn't the local result, that all jobs are started within alpha seconds--quite the contrary, they are almost guaranteed not to unless they can all fit into one node. This isn't a new problem: the old scalers had the exact same behavior, but ironically it didn't really matter because they couldn't scale down very well. We should solve this properly at some point, but for now I recommend increasing the scaler interval.
The beta parameter has always annoyed me a bit: I like the basic idea, but I think the current behavior is less useful than it could be, and can sometimes lead to problematic under-provisioning. Right now it is basically used to ignore small fluctuations, which is fine. But the problem I usually have with the scaler isn't with small fluctuations, it's with fluctuations that are too large, too fast. Usually this happens when thousands of small jobs are being dumped all at once, right after a series of long jobs, causing the autoscaler to freak out and hit its max -- then it rapidly figures out the jobs are pretty short and adjusts to something smaller than the max within 5 minutes, wasting some money. So I'd prefer the inertia parameter to be something much more like a moving average, which could give the autoscaler a chance to adjust, while still being able to hit the max in a reasonable time. (The moving average could still have some minor fluctuations, so maybe the existing beta parameter behavior would still be needed.)
How should we order the "preference" of nodes to launch? Currently a sort is used which will do the "right" thing in the vast, vast majority of cases--but not all. Should we use the ordering provided by the user, and trust them to do it right? Or should we sort to prioritize the cheapest instances first, which is probably what the user wants?

This is amazing @joelarmstrong , thank you!  Will give a full review tomorrow.

I feel like some of this should be captured in the autoscaler doc instead of lost in a
pR>
Joel Armstrong<EMAIL_ADDRESS>writes:

This is a slight refactoring and retouching of the autoscaler. This fixes
several bugs, some of which are blockers for certain niche use-cases. Notably:
• Services weren't being treated specially by the autoscaler. Services are
one case where the degree of parallelization really, really matters. If you
have multiple services, they may all need to be running simultaneously
before any real work can be done. Previously, if your services had a fairly
short runtime, the autoscaler would believe that it could pack them
serially, one after the other. This would lead to service deadlocks in some
situations.
• New node reservations (reservations are sort of like "time-slices" of
resource usage at a node, so you can find the projected resource usage at
each point in time) weren't split, meaning that a 5-minute job that uses 5
cores would be projected to use 5 cores for the entire 1-hour "wallTime" of
the instance. This means in certain situations, more nodes would be created
than would be optimal.
• A long-running job, which "extends" a node reservation for a very long
time, could end up causing other jobs to be packed into its extended
reservations, even though they may be projected to start several hours in
the future. This ends up underprovisioning nodes if you have a few
long-running jobs and many more short jobs. (I've noticed the old
pre-multiple-node-types scaler underestimating the number of needed nodes
with Cactus, though I don't know whether that's caused by this particular
bug or not.)
• The ignored nodes functionality didn't work at all, or when it did, only
rarely, because the ignoredNodes set was based on IP addresses, and the
Mesos batch system was checking hostnames to see if they were ignored.
• When running services or other long-running jobs, the ignored nodes
functionality would rarely lead to deadlocks, where all N nodes were
ignored despite wanting to scale up to N nodes. Now the scaler will
"unignore" nodes if it changes its mind about scaling up or down.
For context, the original scaler worked like this: take the resource
requirements of the last 10000 completed jobs, attempt to figure out how many
nodes would be able to run those so that all jobs started within an hour. Then
multiply that by the length of the current queue times a constant "alpha
packing" factor. This is nice from a performance point of view (because it only
needs to pack at most 10000 jobs), but leads to some really weird results,
because it used the past to predict the future. It also isn't ideal when you
absolutely need parallelism (services), because you never want to
under-provision like you would with an alphaPacking < 1.0. The current scaler
is a blend of using "recent" jobs vs. queued jobs, with alphaPacking
determining the mix between the two.
This alters the scaler so that it works entirely on the basis of queued jobs,
and the "alpha" parameter is a time. The packing attempts to spawn enough nodes
that all the queued jobs will be started by that time. Intuitively this is just
a parameter which controls the degree of parallelization, without (hopefully)
causing deadlocks if it can help it.
Anyway, this has been mangled a bit since I last tested it thoroughly, because
it's been through a rebase and now selectively cherry-picked off my fork. So
it's not ready to merge just yet, but it should be relatively OK.
Some considerations:
• The scaler basically attempts to set the rate of job completion (assuming
homogeneous jobs for simplicity) to n/a, n being the number of jobs and a
being the alpha factor. But the scaler will run again in (by default) 60
seconds, and assuming some jobs have been completed, it will try to set the
rate of job completion lower (because n is lower). The total number of jobs
remaining, n, will decrease from the starting point N as roughly n = N/(t/a
+ 1). So the end result isn't the local result, that all jobs are started
within alpha seconds--quite the contrary, they are almost guaranteed not to
unless they can all fit into one node. This isn't a new problem: the old
scalers had the exact same behavior, but ironically it didn't really matter
because they couldn't scale down very well. We should solve this properly
at some point, but for now I recommend increasing the scaler interval.
• The beta parameter has always annoyed me a bit: I like the basic idea, but
I think the current behavior is less useful than it could be, and can
sometimes lead to problematic under-provisioning. Right now it is basically
used to ignore small fluctuations, which is fine. But the problem I usually
have with the scaler isn't with small fluctuations, it's with fluctuations
that are too large, too fast. Usually this happens when thousands of small
jobs are being dumped all at once, right after a series of long jobs,
causing the autoscaler to freak out and hit its max -- then it rapidly
figures out the jobs are pretty short and adjusts to something smaller than
the max within 5 minutes, wasting some money. So I'd prefer the inertia
parameter to be something much more like a moving average, which could give
the autoscaler a chance to adjust, while still being able to hit the max in
a reasonable time. (The moving average could still have some minor
fluctuations, so maybe the existing beta parameter behavior would still be
needed.)
• How should we order the "preference" of nodes to launch? Currently a sort
is used which will do the "right" thing in the vast, vast majority of
cases--but not all. Should we use the ordering provided by the user, and
trust them to do it right? Or should we sort to prioritize the cheapest
instances first, which is probably what the user wants?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
You can view, comment on, or merge this pull request online at:
https://github.com/BD2KGenomics/toil/pull/2114
Commit Summary
• Refactor bin-packing algorithm
• Fix a bug in bin-packing where new node reservations weren't split
• Fix bin-packing sometimes overpacking a single node for days
• Additional cleanup of the bin-packing code
• Ignore recent jobs entirely when bin-packing
• Don't attempt to prioritize low memory node types
• Fix the scaler leaving your cluster idle if it has connection issues
• Pack jobs only into the first instance that can fit them
• Set the estimated runtime for service jobs to 1 day
• More robust scaler thread
• Fix typo of log object in cluster-scaler
• Remove some of the more verbose debug prints from the scaler
• Move the processRemovedJob definition out of the tight loop
• Unignore nodes if attempting to scale back up
• Remove RecentJobShapes
• Update docstrings and remove excess whitespace
• Parameterize target time for scaler
File Changes
• M src/toil/common.py (15)
• M src/toil/leader.py (17)
• M src/toil/provisioners/clusterScaler.py (568)
• M src/toil/test/provisioners/clusterScalerTest.py (62)
Patch Links:
• https://github.com/BD2KGenomics/toil/pull/2114.patch
• https://github.com/BD2KGenomics/toil/pull/2114.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.*

So I really like a lot of the changes made.  Some points I'd like to address:

I'd like more testing for cases where we have a known desired outcome.  Particularly, the new way that AWS bills (per minute) makes the old way of prioritizing wallTime to maximize the hour that was paid for doesn't make as much sense anymore.  My preference would be launching jobs in parallel as aggressively as possible and almost ignoring the wallTime blocks.  I'm not sure if 60 seconds has been tested as a default wallTime, but I think that would make more sense now (to me at least).
Increasing the 60 second intervals for the scaler interval seems sensible to me.  Should we double it?
Should we add a constant to the service job time rather than simply alphaTime * 24?  I'm just worried the user will set alphaTime to zero, and then have unexpected deadlocks.  I wouldn't mind having a unittest for this particular case.
Should the --betaInertia parameter be a user option?  I agree with your change, though it sounds more like a feature that should be behind the scenes rather than the "try it and see if it works" tweaking that each new user might have to go through.  I'd like to implement the better compensation for rapid swings you're suggesting though I feel like it might be easier if the algorithm in general were reworked to be simpler and just launch jobs as finishing jobs initiate the queue without so much importance on packing based on these time intervals (again, on a pay-by-the-hour system, this made more sense, but I feel like it's less important now).
I agree that it would be pretty great to have an option that would sort by price, and have the workflow use the cheapest instances (especially for, I assume rare, cases where a non-preemptable is cheaper than a preemptable instance), but since these fluctuate, I feel like implementing this would be more of a pain than it's worth.  Allowing the user to provide a price list (which takes the burden off of toil accidentally mispricing something) and prioritize based on this might be a feature worth having (especially for long running, expensive runs like cactus), though I'm not sure it's a priority feature right now.  I like the general idea that jobs should run on the instance with the fewest resources meeting their requirements, but you're right that while generally that's cheaper, it's not always the case, especially when certain instances aim to be cpu "cheap" or memory "cheap".

I like a lot of the commenting, and it's much clearer than it once was.  The NodeReservation object augmentation was pretty great as well.  I agree with Mark that some of these questions and the design choices made should be included in some of the comments as well.  This is really awesome overall and thank you for these changes!

OK, after an additional refactor I'm handing this off to Lon now, who graciously volunteered to shepherd this the rest of the way. It's totally possible that I've broken things a bit with these additional changes (though there are a few more tests that actually test things now). Adding more tests revealed a few bugs, which are fixed now, but I assume are around on master branch as well.
I forgot that in the original PR, I had actually removed betaInertia (because the way it's formulated caused unrecoverable service-deadlock problems by under-provisioning nodes). I added this back in and converted it to an exponentially weighted moving average. I have no idea if having betaInertia actually a good idea or not, but IMO it's a better solution than the old betaInertia regardless--I'll try a run over the weekend and see how it feels.
Lon, I didn't get around to everything we agreed was a good idea yesterday (alphaTime is still called alphaTime, for one). Feel free to change/rename whatever you feel like, I don't have any particular preference. I think for sanity's sake the core algorithm should remain the same, and that alphaTime should remain (gut feeling) above 10min, but otherwise go nuts. We can discuss the alpha parameter and the core algorithm more widely to get other people's input--I think it's a useful discussion to have, because the current algorithm is obviously sub-optimal. But better to get the incremental improvements in even if in the future the core deserves a rethink. I think it's easily possible that betaInertia should be set to 0.0 by default, or just plain removed, I'm not too attached to it. Please let me know if you have questions, sorry for dumping a giant PR on you!

Thanks Joel!  Still in progress and trying to make the tests better.  Fixed one bug at least.

d'oh! thank you! I don't use the clusterStats stuff myself so that totally slipped by me.

I think this is ready to be reviewed and merged.

Added some additional tests.
Added a constant to the service job handling so that it expects service jobs to always take at least an hour.
Scaled down the 1800 self.targetTime default to 1200.
Bumped the self.scaleInterval default up to 60 from 30 as per earlier comments (though should we go above 60?  Maybe I misunderstood this since the comment was that the current default was 60).
A betaInertia of 1.0 smoothed too much and the clusterScaler would never actually change the scaling, allowing things to seemingly run forever, so I set it so that betaInertia must be a value between 0 and 0.9.
Changed the algorithm slightly with Joel to account for tests not provisioning t2.micro tests as expected.
Saved some minutes on jenkins by reducing a time.sleep in one of the tests.
Added the humanize stuff in from bd2k since I was using the human2bytes function.
General commenting.

Overall, thanks for letting me add a little to your PR Joel!  The refactoring you did looks great, and I'm really glad to have these fixes in toil master!
One other thing, some of the comments here are valuable but perhaps not useful to have in our user docs.  I'm going to try to capture it in a design doc on the toil github wiki if that's appropriate.

I think wiki documentation would be great. I agree it doesn't really fit into the user docs well, at least not within the ReadTheDocs framework.
Oops, my bad about the scaleInterval. I thought the default was 60 for some reason, but I never actually checked to see if that was accurate.

Problems should be addressed.  Thanks for catching those and for letting me contribute!  I saw this passed all tests on the last one, and I haven't seen that in a while.  I think this PR is much needed and really appreciate the time you put into submitting it!  Hopefully the scale test goes well.

Thanks! I have a large dataset to run now anyway, so I'll try it with this and see if I can catch anything.

Hey, sorry for the radio silence on this front. I struggled a bit to avoid deadlocks when running from this branch. The good news is that the autoscaler works just fine now, it seems (thanks for the bugfixes!).
But I'm having two issues that cause most runs to stall within ten hours or so:

a cPickle deadlock bug, where occasionally a _toil_worker process will completely stall trying to unpickle the Job object (likely only if the pickle is larger than 4Kb).
Docker daemon crashes causing all Docker runs / execs to stall. (This may be something you only run into with usage of stdin through an attached docker run process.)

I haven't run from upstream toil in a long time so I didn't really notice these before. Have you guys run into any of this during your scale testing? Any unexplained stalls where nothing is happening? I especially think you may have run into 1) before, it seems to happen once in every few thousand jobs for me. But since you are running Cactus every so often, have you noticed 2) too?

Sorry for the silence as well.  I was at the hackathon Mon-Wed and forgot to check this.
Ah, are you using the previous commit?  I'm guessing this is fixed in your fork, but there appears to be an undefined variable currently in the PR called memoryOverhead that I assume should be some percentage of the total memory available: https://github.com/BD2KGenomics/toil/blob/f32fd565f324e74980dc163af953b5334df02e0f/src/toil/provisioners/aws/awsProvisioner.py#L307
I've run 1000 jobs for integration testing on master only, but these were simple, time.sleep(#) jobs that I created for this explicit purpose, and we've only started doing this in the past month or so.  Because of how the current autoscaler functions, I had been hoping to have this PR implemented before continuing to develop more complex runs.  Running 1000+ jobs is pretty new, so I can't say we've run into either problem before.
As for our small pestis cactus runs, they are run pretty infrequently since they're not part of the main unit tests, and take far less than 10 hours.  I don't think we currently test anything right now that takes 10 hours.  As for this particular branch, the only integration test has been your cactus run.

Oops, my bad about the memoryOverhead thing. That is from a different piece of work, trying to solve the problem of the autoscaler and Mesos disagreeing about available RAM. (Long story. Mesos reserves a hardcoded 1GB of ram for some reason, which is a really big problem on small instances.)
Ah, OK. That makes sense. I can push a fix for the pickle deadlock. I've tracked it down and unfortunately it's a bug in cPickle. There isn't much we can do to avoid it aside from never using pickle.load from large, remote jobStore streams.
The docker issue is a bit weird. I can't tell exactly what the deal is, but it doesn't seem like it's really toil's problem. I can work around it on the Cactus side, it may be that we are abusing too many weird, uncommon docker use cases, and that some docker bug was introduced recently.

Yes, I remember seeing that.  I think you might have even been the one to show me.  Seems like a necessary thing to correct for.
Is this link the cPickle error?  It seems as though normal pickle works if this is the case, and it's only cPickle that breaks.  Is that the plan of attack?  :)
I'm unclear about the docker error.  Are these the worker mesos images themselves or docker images called from within the mesos images or everything?  I wouldn't mind having a look because now you have me curious.

I'm not sure it's a bug that anyone else but toil could run into. It has to do with cPickle occasionally not releasing the GIL when reading. (C extensions must release the GIL before a potentially blocking operation, for exactly the reason we are having problems with: the operation may be relying on a separate Python thread to run and provide the data.) Switching to HIGHEST_PROTOCOL means that it releases the GIL the vast majority of the time, but sadly I discovered there are still sometimes a few  read_global opcodes that get dumped, and the data for those is read without releasing the GIL. So we are inevitably stuck if we try to load from a stream.
(We could switch to pickle rather than cPickle to avoid this. But that has performance implications, and Python 3 aliases pickle to cPickle anyway.)
Yeah, these are issues with docker images called from within the toil_worker container. It's a bit difficult to track down, but I did get a few kernel errors about dockerd filesystem sync syscalls getting stuck for several minutes. So it may be a kernel issue, or just some weird coincidental interaction with EBS.

I see.  So is the fix to stream the pipe to a file before running cPickle.load?
(Makes sense, and I think we can't move to py3 soon enough.)
Huh, that's pretty interesting.  If you have the errors, could you send them my way?

Here's the hang. I got quite a few of these, which roughly corresponded to the time where docker run commands seemed to start hanging, though I'm not positive about that.
[31089.547240]       Not tainted 4.14.16-coreos #1
[31089.549550] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[31089.553487] dockerd         D    0  1720      1 0x00000080
[31089.556215] Call Trace:
[31089.557493]  ? __schedule+0x2e3/0x890
[31089.559328]  schedule+0x32/0x80
[31089.560800]  jbd2_log_wait_commit+0x93/0x110 [jbd2]
[31089.563393]  ? wait_woken+0x80/0x80
[31089.565115]  ext4_resize_fs+0x8f50/0xe390 [ext4]
[31089.567393]  0xffffffffc03d8233
[31089.568972]  sync_filesystem+0x6a/0x90
[31089.570847]  generic_shutdown_super+0x22/0x120
[31089.573127]  kill_anon_super+0xe/0x20
[31089.574782]  deactivate_locked_super+0x39/0x70
[31089.577020]  cleanup_mnt+0x3b/0x70
[31089.578799]  task_work_run+0x80/0xa0
[31089.580565]  exit_to_usermode_loop+0xa4/0xb0
[31089.582786]  do_syscall_64+0xe9/0x1c0
[31089.584690]  entry_SYSCALL64_slow_path+0x25/0x25
[31089.587035] RIP: 0033:0x495e24
[31089.588630] RSP: 002b:000000c4248d0eb0 EFLAGS: 00000246 ORIG_RAX:<PHONE_NUMBER>0000a6
[31089.592223] RAX:<PHONE_NUMBER>000000 RBX:<PHONE_NUMBER>000000 RCX:<PHONE_NUMBER>495e24
[31089.595714] RDX:<PHONE_NUMBER>000000 RSI:<PHONE_NUMBER>000002 RDI: 000000c426163420
[31089.599226] RBP: 000000c4248d0f10 R08:<PHONE_NUMBER>000000 R09:<PHONE_NUMBER>000000
[31089.602722] R10:<PHONE_NUMBER>000000 R11:<PHONE_NUMBER>000246 R12:<PHONE_NUMBER>000000
[31089.606311] R13: 00000000ffffffee R14:<PHONE_NUMBER>001420 R15:<PHONE_NUMBER>000049

Looks like it's something to do with syncing the filesystem, though it would be interesting to know exactly what syscall it's hanging on from Docker. It could be some weird problem caused by mounting /var/lib/docker in the containers, given that it appears to be something to do with mounting/unmounting filesystems. But I'm not sure I totally trust the stacktrace: why would ext4_resize_fs get called?
I also had the journal crash for some reason:
[39528.694227] systemd-coredump[67557]: Coredump diverted to /var/lib/systemd/coredump/core.systemd-journal.0.296fcc15f88b46bab231ebcc5
63f456d.956.1522843727000000.lz4
[39528.695542] systemd-journald[67633]: File /var/log/journal/6ed0fa8a724b40a0b4c57e5d4f7724c9/system.journal corrupted or uncleanly sh
ut down, renaming and replacing.
[39528.730025] systemd-journald[67633]: Received SIGTERM from PID 1 (systemd).```

Yes, reading the stream into a file is the solution I had in mind. I have a patch which seems to fix the immediate problem. I can do a PR tonight or early tomorrow.

What's the status for this? @glennhickey @adamnovak are having autoscaler issues with multiple node types that is somewhat blocking and at least slowing down vg testing. @DailyDreaming Is this is the current sprint?

I forget the status of this. I think at one point I was waiting for the provisioner overhaul to get committed. But I ran the 48-way on this, or something close to it, and really the only issues I ran into were docker bugs.
I can rebase and remove the conflicts, give me an hour or two. Hopefully the tests will pass; i don't think there were really any remaining issues with this PR.

This would be really handy for toil-vg and would resolve #2195.
On Mon, May 7, 2018 at 2:34 PM, Joel Armstrong<EMAIL_ADDRESS>wrote:

I forget the status of this. I think at one point I was waiting for the
provisioner overhaul to get committed. But I ran the 48-way on this, or
something close to it, and really the only issues I ran into were docker
bugs.
I can rebase and remove the conflicts, give me an hour or two. Hopefully
the tests will pass; i don't think there were really any remaining issues
with this PR.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/BD2KGenomics/toil/pull/2114#issuecomment-387160346,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA2_7uJB06D2uDs_2NYlaE5N11n32Jkcks5twJOvgaJpZM4SiHB4
.


refactor: Remove redundant mat test

Ref #71
Description
Removing redundant materialisation which is already implemented on the dbt-core side.
Checklist

[x] I have run this code in development and it appears to resolve the stated issue.
[x] This PR includes tests, or tests are not required/relevant for this PR.
[ ] I have updated CHANGELOG.md and added information about my change.
[x] I have removed any print or log calls that were added for debugging.
[ ] If this PR requires a new PyPI release I have bumped the version number.
[x] I have verified that this PR contains only code changes relevant to this PR.
[x] If further integration tests are now passing I've edited tests/functional/adapter/test_basic.py to account for this.
[x] I have pulled/merged from the main branch if there are merge conflicts.
[x] After pulling/merging main I have run pytest on the included or updated tests/functional/adapter/test_basic.py.

Verified via tests and jaffle shop (test) run.


add metric acm_managed_cluster_worker_cores

The related issue: https://issues.redhat.com/browse/ACM-11236

/unhold


Fix dependency for speed/distance

Didn't see https://github.com/hcrohland/waterrower-ble/tree/hcr 💯


WIP: Allow users to control which groups a client is added to

This PR fixes #6 by allowing users to manually control which Channel Layers groups a subscribed client is added to, and which groups subscription events are broadcast over.
The consumer instance is now passed to each subscription resolver as a the root value, and exposes a function called subscribe. When called, this function will add the currently connected client to the specified group and return an Observable of all values sent to that group.
A model mixin called SubscriptionModelMixin has been created, which uses django-lifecycle instead of Django signals to automatically trigger subscription events when model instances are created, updated, or deleted.
This PR also removes the legacy SubscriptionEvent and ModelSubscriptionEvent classes, and removes the default post_save_subscription and post_delete_subscription signal handlers, as they are no longer used.
Example:
# your_app/graphql/subscriptions.py
class MessageCreatedSubscription(graphene.ObjectType):
    message_created = graphene.Field(MessageType)
    message_updated = graphene.Field(MessageType, id=graphene.String())

    def resolve_message_created(root, info):
        return root.subscribe("messageCreated")

    def resolve_message_created(root, info, id):
        return root.subscribe(f"messageCreated.{id}")

# your_app/models.py
from django.db import models
from graphene_subscriptions.models import SubscriptionModelMixin

class Message(SubscriptionModelMixin, models.Model):
    # ...

This PR also fixes #8 and #2.

This PR also fixes #8 and #2.

@jaydenwindle Will this get released any soon? This will come handy in a project I am working on.

Hey @oldani! Yes, I'm planning to merge this PR and release v2 as soon as I get a chance to write a v1 -> v2 migration guide for the docs, since there are several breaking changes. I've been a bit busy with other projects recently so haven't had time to finish it.
If you'd like to help with the migration guide I'd definitely accept a PR!


java.io.IOException: setDataSource failed.??

http://i.prntscr.com/sZLwiZwATYa099EuuVQjzw.png
Am getting alway this error.how can i solve it.?
My URL:
https://www.emenator.com/flash/modules/mp3/get_mobile.php?id=558

Please provide a valid URL like 'http://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3' which is access able by Android MediaPlayer. I know URL is valid if you open in chrome browser it's playing that song but it's not access able by Android Media Player class, this not a valid data source to play. That's why you are getting error in your console(IOException setDataSource failed).

Can you please tell us more about how this url should be constructed, in such a way that it supports the media player?

Hi prakashmp,
Try to create URL path as end with your music file extension like "...../music/mymusic.mp3" then only it will be readable by android Media Player. If you relay stay with your URL than you can try https://github.com/iammert/RadioPlayerService this lib also. I think it can also help you.

okay i will check it..thank you buddy :) dono its working or not.? i struggle one week in stream audio player.:(


Added  Swap_kth_elements.cpp

This file contains O(1) solution for the problem Swap_kth_elements

According to the guidelines, kindly get this issue assigned first mentioning your language of preference.


Store "cold" keys on disk

etcd cannot store more keys than it has memory for. This was an important "feature" in the begging so we could focus on consensus and the API - the absolute hardest parts to get right in projects like etcd.
This is now a bug. We have been evaluating options and plan to land this feature in 0.5.

Addressed by the v3 backend design doc.


sudo brew services list unclear error message

Clean install on clean macOS 13.1
sudo brew services list dosn't work:
>sudo brew services list
Error: Running Homebrew as root is extremely dangerous and no longer supported.
As Homebrew does not drop privileges on installation you would be giving all
build scripts full access to your system.
Error: Failure while executing; `/opt/homebrew/bin/brew tap homebrew/services` exited with 1.

Running sudo brew services list executes two internals commands: services and tap. The first command works properly, but the second produce the error. Could you please fix the bug in the script?
https://github.com/Homebrew/brew/blob/884c4bec14073314ac6e8e9a6a64bc3b26161ab6/Library/Homebrew/brew.sh#L178-L196

Workaround: Run brew services list first, to tap the Homebrew services code tap as you. Once that's done sudo brew services list should work as advertised.

Thanks. My own way is executing brew tap homebrew/services before sudo brew services list.
However, this place make a lot of bad experience because no correct error message about whats was wrong.

Sure, but such fixes are only available to most (read: non-dev-mode) users on the next official release. My suggestion is to solve your (and whoever else ran into the same) problem now.


notifyPath should re-set root-properties when the binding target does not support notifyPath

Description
When handling path notifications, notifyPath should set root-properties when the binding target does not support notifyPath.  There are 2 cases where a path notification currently gets dropped on the floor:

The binding target is not a Polymer element and thus does not support path notifications, e.g. <a href="/list?items={{items}}>, where items is e.g. an Array that should be stringified
The binding target is a Polymer element, but the binding is to a property that doesn't have property effects, e.g. <my-element>{{items}}</my-element>, where items is an Array that should be stringified into textContent

Live Demo
https://jsbin.com/coluvaw/1/edit?html,output
Steps to Reproduce

open link
click "add friends" button

Expected Results
0,1,2,3,4
Actual Results
0,1,2,3
Browsers Affected
All
Versions

Polymer: v1.6

Poke. Any plans to implement this?

Closing as this appears to be fixed in Polymer 2: https://jsbin.com/yuxupefone/edit?html,console,output


Add some way to generate OpenAPI

This will be fire, clients generation, documentation, etc. Is there anything I can do to accelerate this?

Something like this https://github.com/swaggo/echo-swagger

bunrouter respect net/http so you can use every swagger middleware


Implement a way to validate completed block upload

Currently when we upload blocks to the object storage, if we encounter an error we clean up after ourselves and delete all files that were uploaded so far. This is however not 100% safe yet.
If the uploader has a hard crash, orphaned objects are left behind. Files are uploaded in order so a block will only be validly considered if the meta.json exists, which is uploaded last. This guards against for example, considering a block during garbage collection even though it is only partial. This is however implicit and makes us rely on upload order, i.e. we cannot parallelise it.
We should come up with a way to ensure this. This could either be a dedicated done file for each block or setting of a metadata/label onto the meta.json file in the object storage. Either way, our code would do an explicit finalization after successful upload of all block data.
We should also add a background cleanup procedure that detects and purges partial blocks.

https://github.com/improbable-eng/thanos/pull/295 fixes it I believe.
We explicitly ensure that meta.json is uploaded as a last item. It seems like a good enough solution for now.


pull request for e2e test at 11/30/2016 10:26:05

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Scaffold inserts custom theme for persistentFooterButtons.

What is the intention that the ButtonTheme for items in persistentFooterButtons of a Scaffold is overwritten by it?
Example
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData(
        primaryColor: Colors.green,
        accentColor: Colors.redAccent,
        buttonTheme: ButtonThemeData(
          buttonColor: Colors.lightBlue,
          textTheme: ButtonTextTheme.primary,
        ),
      ),
      home: Scaffold(
        persistentFooterButtons: <Widget>[
          RaisedButton(
            onPressed: () => print("pressed button"),
            child: Text("PRESS"),
          ),
        ],
      ),
    );
  }
}

In this case I would expect that the color set in the theme with ButtonTextTheme.primary is used as the text color for the button. (In this case the primary color green and not redAccent)
But in reality the theme is overwritten by ButtonTheme.bar in the Scaffold.
To get your own theme back you need to wrap it inside another ButtonTheme widget like so:
ButtonTheme.fromButtonThemeData(
    data: Theme.of(context).buttonTheme,
    child: RaisedButton(
        onPressed: () {},
        child: Text("PRESS"),
    ),
)

This feels very unintuitive and I wonder what it is good for and why it is not left to the user?
Logs
[✓] Flutter (Channel beta, v1.7.8+hotfix.4, on Linux, locale en_GB.UTF-8)
    • Flutter version 1.7.8+hotfix.4 at /home/martinniederl/development/flutter
    • Framework revision 20e59316b8 (4 weeks ago), 2019-07-18 20:04:33 -0700
    • Engine revision fee001c93f
    • Dart version 2.4.0

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /home/martinniederl/Android/Sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /home/martinniederl/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b16-5323222)
    • All Android licenses accepted.

[✓] Android Studio (version 3.4)
    • Android Studio at /home/martinniederl/android-studio
    • Flutter plugin version 37.1.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b16-5323222)

[✓] VS Code (version 1.36.1)
    • VS Code at /usr/share/code
    • Flutter extension version 3.3.0

[✓] Connected device (1 available)
    • SM A500FU • 951ad3cd • android-arm • Android 5.0.2 (API 21)

• No issues found!

Issue still exist
[✓] Flutter (Channel dev, v1.18.0, on Mac OS X 10.15.4 19E266, locale en-GB)
    • Flutter version 1.18.0 at /Users/taha/Code/flutter_dev
    • Framework revision 14f3a36aba (3 days ago), 2020-04-03 17:06:01 -0700
    • Engine revision 09bc1fc45e
    • Dart version 2.8.0 (build 2.8.0-dev.20.0 e736495eb7)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /Users/taha/Code/sdk
    • Platform android-29, build-tools 29.0.3
    • ANDROID_HOME = /Users/taha/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_212-release-1586-b4-5784211)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.4, Build version 11E146
    • CocoaPods version 1.9.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 3.6)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 45.0.1
    • Dart plugin version 192.7761
    • Java version OpenJDK Runtime Environment (build
      1.8.0_212-release-1586-b4-5784211)

[✓] VS Code (version 1.43.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.9.0

[✓] Connected device (4 available)
    • SM M305F   • 32003c30dc19668f • android-arm64  • Android 10 (API 29)
    • macOS      • macOS            • darwin-x64     • Mac OS X 10.15.4 19E266
    • Chrome     • chrome           • web-javascript • Google Chrome
      80.0.3987.149
    • Web Server • web-server       • web-javascript • Flutter Tools

• No issues found!

Hi @MartinNiederl
ButtonTheme is now obsolete, please use ElevatedButton, it works fine with persistentFooterButtons

code sample
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Material App',
      theme: ThemeData().copyWith(
        primaryColor: Colors.green,
        accentColor: Colors.redAccent,
        elevatedButtonTheme: ElevatedButtonThemeData(
          style: ButtonStyle(
            backgroundColor: MaterialStateProperty.all<Color>(Colors.cyan),
          ),
        ),
      ),
      home: Home(),
    );
  }
}

class Home extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: ElevatedButton(
          onPressed: () => print("pressed button"),
          child: Text("PRESS"),
        ),
      ),
      persistentFooterButtons: <Widget>[
        ElevatedButton(
          onPressed: () => print("pressed button"),
          child: Text("PRESS"),
        ),
      ],
    );
  }
}

multiple flutter doctor -v
[✓] Flutter (Channel stable, 1.22.5, on macOS 11.1 20C69 darwin-x64, locale
    en-GB)
    • Flutter version 1.22.5 at /Users/tahatesser/Code/flutter_stable
    • Framework revision<PHONE_NUMBER> (12 days ago), 2020-12-10 11:54:40 -0800
    • Engine revision ae90085a84
    • Dart version 2.10.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/tahatesser/Code/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/tahatesser/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Volumes/Extreme/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.0

[!] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.52.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.17.0

[✓] Connected device (2 available)
    • RMX2001 (mobile)     • EUYTFEUSQSRGDA6D          • android-arm64 • Android
      10 (API 29)
    • Taha’s iPad (mobile) • 00008020-000255113EE8402E • ios           • iOS
      14.3

! Doctor found issues in 1 category.

[✓] Flutter (Channel dev, 1.26.0-1.0.pre, on macOS 11.1 20C69 darwin-x64, locale
    en-GB)
    • Flutter version 1.26.0-1.0.pre at /Users/tahatesser/Code/flutter_dev
    • Framework revision 63062a6443 (9 days ago), 2020-12-13 23:19:13 +0800
    • Engine revision 4797b06652
    • Dart version 2.12.0 (build 2.12.0-141.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/tahatesser/Code/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/tahatesser/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Volumes/Extreme/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[!] Android Studio
    • Android Studio at /Applications/Android/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Android Studio not found at /Applications/Android/Contents
    • Try updating or re-installing Android Studio.

[✓] VS Code (version 1.52.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.17.0

[✓] Connected device (4 available)
    • RMX2001 (mobile)     • EUYTFEUSQSRGDA6D          • android-arm64  •
      Android 10 (API 29)
    • Taha’s iPad (mobile) • 00008020-000255113EE8402E • ios            • iOS
      14.3
    • macOS (desktop)      • macos                     • darwin-x64     • macOS
      11.1 20C69 darwin-x64
    • Chrome (web)         • chrome                    • web-javascript • Google
      Chrome 87.0.4280.88

! Doctor found issues in 1 category.

Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


Prevent Homeassistant to create a segfault with OZW

Description:
If user provided an out of bound value for list config parameters OZW would crash, and taking HA with it in the process.
This change should stop that behaviour by checking how many available items there are before executing the set_config_parameter service.
Related issue (if applicable): fixes #5067

I've tested it on my own setup, but would appreciate if someone else could verify that it works perfectly.


[FSDE-4222] Chevrons in data table mobile view must be clickable

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/2/. :white_check_mark:

You can check this PRs instance in https://4ef4d9b6fa357947245d61bb3a00bbba87f69f32-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/3/. :white_check_mark:

You can check this PRs instance in https://b3e2d60027ed3b4b57b70ec322a0b951861c4a3f-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/4/. :white_check_mark:

You can check this PRs instance in https://b1b28fac7c323ca61ea02cc015fcea241e439ff6-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/5/. :white_check_mark:

You can check this PRs instance in https://599c60cfcceacbe2f9b54f2fe4e50168ac523787-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/6/. :white_check_mark:

You can check this PRs instance in https://19ad440b169887733e6410d64bac7078be62cdd1-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/7/. :white_check_mark:

You can check this PRs instance in https://4e6e8d5daf80f6da8fa9ec0f8b247a64b7c3021a-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/8/. :white_check_mark:

You can check this PRs instance in https://52e61dc901680040bc4fec26ee022de54d7ff085-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/9/. :white_check_mark:

You can check this PRs instance in https://fc830032d56dfa224dcbf657fcd0b5218b6a0225-chi-pr.enterprise.ctl.io

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-165/10/. :white_check_mark:

You can check this PRs instance in https://8dcba0c24677a044f245e41cb59d84e69471639a-chi-pr.enterprise.ctl.io


Add support for update property

If Include is not set, try Update
Fixes #170

Hi @daveaglick,
Any chance you will have some time to look at the PR's?

Probably not this week (I'm on vacation and try hard to unplug), but I'll definitely carve out time early next week when I'm back. Got a few projects with backed up PRs to look at.

No problem, enjoy your vacation!

*poke*
@daveaglick, consider this a friendly nudge :)

Guess it's finally time I clear the backlog here :) I'll start going through the open PRs and merging them with a release once they're all done.

And merged! Thanks...and especially for including some tests that make this easy to hit the merge button :). I'll get a release out ASAP once I clear the rest of the backlog.


Content of exception_messages string pointer arrays is empty in 18-interrupts

When learning 18-interrupts, content of exception_messages in cpu/isr.c is empty. All items are 0x0. It seems that ld don't replace the string constants with real address.
I do this tutorial on Gentoo with gcc (Gentoo 7.3.0-r3 p1.4) 7.3.0, GNU ld (Gentoo 2.30 p5) 2.30.0.
CFLAGS is -m32 -fno-pie -g. There would be errors if I don't use -fno-pie.

I had this same error and fixed it by loading more sectors from disk at boot time. I originally was loading 8 into dh, but I changed it to 15 and it worked.
load_kernel:
  mov bx, MSG_LOAD_KERNEL ; Print message to say loading kernel
  call print_string
  call print_nl

  mov bx, KERNEL_OFFSET   ; Read from disk and store in 0x1000
  mov dh, 15
  mov dl, [BOOT_DRIVE]  ; From boot disk
  call disk_load
  ret

I had this same error and fixed it by loading more sectors from disk at boot time. I originally was loading 8 into dh, but I changed it to 15 and it worked.
load_kernel:
  mov bx, MSG_LOAD_KERNEL ; Print message to say loading kernel
  call print_string
  call print_nl

  mov bx, KERNEL_OFFSET   ; Read from disk and store in 0x1000
  mov dh, 15
  mov dl, [BOOT_DRIVE]  ; From boot disk
  call disk_load
  ret

Yeah, when the kernel gets too big it doesn't load the whole kernel so parts of it are missing and so it bugs out and sometimes crashes. I ran out of space on my extension to the kernel so I had to make it as large as it could be, but then I ran out of space so now im moving to grub for more kernel space

Use grub guys, not only will your operating system be more universal, but it will also not have to worry about booting up the computer

Just make sure if you post your code on github to include a tarball that you built grub from so you comply with the license for grub

??
Does the grub license say anything about including a tarball of it?
Can't you just install grub and use it?

Well on the OS dev barebones tutorial wiki page, it says you have to have a tarball of the grub you built in your source somewhere if you are going to share it


The package doesn't work with more recent versions of the azure package

Currently this package depends on:
"azure": ">= 0.7.2",
But the newer versions of azure have an API change that breaks it (Specifically, the method BlobService::createBlockBlobFromFile has been removed.
Last known good is azure 0.8.1.

I downgraded back to 0.8.1 and it works. Temporary fix.

Thanks @mwmwmw


soft-serve: 0.3.2 -> 0.3.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/charmbracelet/soft-serve/releases.
meta.description for soft-serve is: "A tasty, self-hosted Git server for the command line"
meta.homepage for soft-serve is: "https://github.com/charmbracelet/soft-serve"
meta.changelog for soft-serve is: "https://github.com/charmbracelet/soft-serve/releases/tag/v0.3.3"
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.3.3 with grep in /nix/store/d96a5306x8ha2zrfmi3ysm2rn7k94hnw-soft-serve-0.3.3
found 0.3.3 in filename of file in /nix/store/d96a5306x8ha2zrfmi3ysm2rn7k94hnw-soft-serve-0.3.3

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
soft-serve

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/d96a5306x8ha2zrfmi3ysm2rn7k94hnw-soft-serve-0.3.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A soft-serve https://github.com/r-ryantm/nixpkgs/archive/118357114093fb326b2bd116fdcb189db70de3cf.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/d96a5306x8ha2zrfmi3ysm2rn7k94hnw-soft-serve-0.3.3
ls -la /nix/store/d96a5306x8ha2zrfmi3ysm2rn7k94hnw-soft-serve-0.3.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

soft-serve

Maintainer pings
cc @penguwin for testing.

Result of nixpkgs-review pr 185269 1

1 package built:

soft-serve


CircleCI write rights for me

@conorhastings could you give me write rights on circleCI so I can rerun builds?
The failure on: https://github.com/conorhastings/react-syntax-highlighter/pull/155 looks like a npm problem but I can't rerun it.
npm ERR! path /home/ubuntu/conorhastings/react-syntax-highlighter/node_modules/.staging/has-value-addf75b1/node_modules/abbrev
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename
npm ERR! enoent ENOENT: no such file or directory, rename '/home/ubuntu/conorhastings/react-syntax-highlighter/node_modules/.staging/has-value-addf75b1/node_modules/abbrev' -> '/home/ubuntu/conorhastings/react-syntax-highlighter/node_modules/.staging/abbrev-56ec71db'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

Actually probably not needed anymore


Remove usage to ColumnView.repeatStringsSizes

Since the cudf PR https://github.com/rapidsai/cudf/pull/12543 reworks strings::repeat_strings so that it can detect output overflow, ColumnView.repeatStringsSizes is no longer needed. This PR removes its usage and will just call ColumnView.repeatStrings directly.
Depends on (so calling to ColumnView.repeatStrings will have the ability to detect output overflow):

https://github.com/rapidsai/cudf/pull/12543

Note that the "breaking change" is causes by https://github.com/rapidsai/cudf/pull/12546. Thus, such "breaking change" does not exist yet until that PR is merged.

build


Make it possible to generate two types of events.

Using the nexmark::EventGenerator::with_type_filter(EventType), it is currently possible to generate one of the following:

All types of events
Only auction events
Only bid events
Only person events

Some nexmark queries require only two types of events, e.g., bids and auctions. Would it be possible to extend the generator so that we can also produce the following?:

Only bid and auction events
Only auction and person events
Only person and bid events

Hmm, is it perhaps possible to get around this by tweaking the person_proportion, auction_proportion and bid_proportion settings?