TrainingWrapper does not support line breaks

Notebook
When training RETRO with the standard methods, TrainingWrapper does not add line breaks to the dataset. This can have a bad effect on many NLP tasks.
Input *.txt:
First Citizen:
We are accounted poor citizens, the patricians good.
What authority surfeits on would relieve us: if they
would yield us but the superfluity, while it were
wholesome, we might guess they relieved us humanely;
but they think we are too dear: the leanness that
afflicts us, the object of our misery, is as an
inventory to particularise their abundance; our
sufferance is a gain to them Let us revenge this with
our pikes, ere we become rakes: for the gods know I
speak this in hunger for bread, not in thirst for revenge.

Second Citizen:
Would you proceed especially against Caius Marcius?

All:
Against him first: he's a very dog to the commonalty.

Model output after traing:
some - - on my head, were even so salts to death strike That which may bet with tears I have found to life, which sweeter than now to dony : be known betwixcombed oaths ring yet in Corioli turnseth from him Dear life redeems doth thinkment for faith ; Or shall be slack than death within this face, PETRUCHIO : Now, wind and house or free thee better now. KATHARINA : Now, in mine honourable fellow : in your chat with me to be it, alive, I think, If to use than my wife, if this rebellious earth Have you will break out The strange s of yours cro

@0x7o ohh interesting, this must be some issue with the default BERT tokenizer, i'll take a look next week

Judging by the warning, the script tones the entire file at once, not in parts
Token indices sequence length is longer than the specified maximum sequence length for this model (4112160 > 512). Running this sequence through the model will result in indexing errors

Judging by the error below, the script handles the file as a whole, not in batches
Token indices sequence length is longer than the specified maximum sequence length for this model (3449121 > 512). Running this sequence through the model will result in indexing errors

And one more error that occurs at the very end of data processing. Before knn
RuntimeError: Error in void faiss::gpu::GpuIndexIVFPQ::verifySettings_() const at /project/faiss/faiss/gpu/GpuIndexIVFPQ.cu:428: Error: 'ivfpqConfig_.interleavedLayout || IVFPQ::isSupportedPQCodeLength(subQuantizers_)' failed: Number of bytes per encoded vector / sub-quantizers (128) is not supported

@lucidrains, tokenizer distorts the text. I think the problem is the difference between bert-cased and bert-uncased
Here is an example of a dataset from the program code:
} \ n else if ( GameManager. _ instance. won & &! GameManager. _ instance. keepPlaying ) { " won " } \ n else { " running " } \ n''') \ n \ n def get _ score ( self ) : \ n return self. execute ('GameManager. _ instance. score') \ n \ n def get _ board ( self ) : \ n # Chrome refuses to serialize the Grid object directly through the debugger. \ n grid = json. loads ( self. execute ('JSON. stringify ( GameManager. _ instance. grid )') ) \ n \ n board = [ [ 0 ] * 4 for _ in range ( 4 ) ] \ n for row in grid ['cells'] : \ n for cell in row : \ n if cell is None : \ n continue \ n pos = cell ['x'], cell ['y'] \ n tval = cell ['value'] \ n board [ pos [ 1 ] ] [ pos [ 0 ] ] = int ( round ( math. log ( tval, 2 ) ) ) \ n \ n return board \ n \ n def execute _ move ( self, move ) : \ n # We use UDLR ordering ; 2048 uses URDL ordering \ n move = [ 0, 2, 3, 1 ] [ move ] \ n self. execute ('GameManager. _ instance. move ( % d )'% move ) \ n \ nclass Keyboard2048Control ( Generic2048Control ) : \ n'''Control 2048 by accessing the DOM and using key events. \ n \ n This is relatively slow, and may be prone to race conditions if your \ n browser is slow. However, it is more generally compatible with various \ n clones of 2048.'''\ n \ n def setup ( self ) : \ n self. execute ( \ n'''\ n var elems = document. getElementsByTagName ('div') ; \ n for ( var i in elems ) \ n if ( elems [ i ]. className = ='tile - container') { \ n tileContainer = elems [ i ] ; \ n break ; \ n } \ n''') \ n \ n def get _ score ( self ) : \ n score = self. execute ('''\ n var scoreContainer = document. querySelector ( ". score - container " ) ; \ n var scoreText ='' ; \ n var scoreChildren = scoreContainer. childNodes ; \ n for ( var i = 0 ; i < scoreChildren. length ; + + i ) { \ n if ( scoreChildren [ i ]. nodeType = = Node. TEXT _ NODE ) { \ n scoreText + = scoreChildren [ i ]. textContent ; \ n } \ n } \ n scoreText ; \ n''') \ n \ n return int ( score ) \ n \ n def get _ board ( self ) : \ n res = self. execute ( \ n'''\ n var res = [ ] ; \ n var tiles = tileContainer. children ; \ n for ( var i = 0 ; i < tiles. length ; i + + ) \ n res. push ( tiles [ i ]. className ) ; \ n res \ n''') \ n board = [ [ 0 ] * 4 for _ in range ( 4 ) ] \ n for tile in res : \ n tval = pos = None \ n for k in tile. split ( ) : \ n m = re. match ( r'^ tile - ( \ d + ) $ ', k ) \ n if m : \ n tval = int ( m. group ( 1 ) ) \ n m = re. match ( r'^ tile - position - ( \ d + ) - ( \ d + ) $ ', k ) \ n if m : \ n pos = int ( m. group ( 1 ) ), int ( m. group ( 2 ) ) \ n board [ pos [ 1 ] - 1 ] [ pos [ 0 ] - 1 ] = int ( round ( math. log ( tval, 2 ) ) ) \ n \ n return board \ n \ n def execute _ move ( self, move ) : \ n key = [ 38, 40, 37, 39 ] [ move ] \ n self. send _ key _ event ('keydown ', key ) \ n time. sleep ( 0. 01 ) \ n self. send _ key _ event ('keyup ', key ) \ n time. sleep ( 0. 05 ) \ n \ nclass Hybrid2048Control ( Fast2048Control, Keyboard2048Control ) : \ n'''Control 2048 by hooking the GameManager and using keyboard inputs. \ n \ n This is safe and fast, and correctly generates keyboard events for compatibility. \ n'''\ n \ n setup = Fast2048Control. setup \ n get _ status = Keyboard2048Control. get _ status \ n get _ score = Fast2048Control. get _ score \ n get _ board = Fast2048Control. get _ board \ n execute _ move = Keyboard2048Control. execute _ move \ n # Preprocess cornell movie dialogs dataset \ n \ nfrom multiprocessing import Pool \ nimport argparse \ nimport pickle \ nimport random \ nimport os \ nfrom urllib. request import urlretrieve \ nfrom zipfile import ZipFile \ nfrom pathlib import Path \ nfrom tqdm import tqdm \ nfrom model. utils import Tokenizer, Vocab, PAD _ TOKEN, SOS _ TOKEN, EOS _ TOKEN \ n \ nproject _ dir = Path ( _ _ file _ _ ). resolve ( ). parent \ ndatasets _ dir = project _ dir. joinpath ('datasets /') \ ncornell _ dir = datasets _ dir. joinpath ('cornell /') \ n \ n # Tokenizer \ ntokenizer = Tokenizer ('spacy') \ n \ ndef prepare _ cornell _ data ( ) : \ n " " " Download and unpack dialogs " " " \ n \ n zip _ url ='http : / / www. mpi - sws. org / ~ cristian / data / cornell _ movie _ dialogs _ corpus. zip'\ n zipfile _ path = datasets _ dir. joinpath ('cornell. zip') \ n \ n if not datasets _ dir. exists ( ) : \ n datasets _ dir. mkdir ( ) \ n \ n # Prepare Dialog data \ n if not cornell _ dir. exists ( ) : \ n print ( f'Downloading { zip _ url } to { zipfile _ path }') \ n urlretrieve ( zip _ url, zipfile _ path ) \ n print ( f'Successfully downloaded { zipfile _ path }') \ n \ n zip _ ref = ZipFile ( zipfile _ path,'r') \ n zip _ ref. extractall ( datasets _ dir ) \ n zip _ ref. close ( ) \ n \ n datasets _ dir. joinpath ('cornell movie - dialogs corpus'). rename ( cornell _ dir ) \ n \ n else : \ n print ('Cornell Data prepared!') \ n \ n \ ndef loadLines ( fileName, \ n fields = [ " lineID ", " characterID ", " movieID ", " character ", " text " ], \ n delimiter = " + + + $ + + + " ) : \ n " " " \ n Args : \ n fileName ( str ) : file to load \ n field ( set < str > ) : fields to extract \ n Return : \ n dict < dict < str > > : the extracted fields for each line \ n " " " \ n lines = { } \ n \ n with open ( fileName,'r ', encoding ='iso - 8859 - 1') as f : \ n for line in f : \ n values = line. split ( delimiter ) \ n \ n # Extract fields \ n lineObj = { } \ n for i, field in enumerate ( fields ) : \ n lineObj [ field ] = values [ i ] \ n \ n lines [ lineObj ['lineID'] ] = lineObj \ n \ n return lines \ n \ n \ ndef loadConvers

@0x7o i think the solution may be to add the newline token https://discuss.huggingface.co/t/feat-tokenizers-how-to-make-models-aware-of-structuring-linebreaks/3711 , however, without training BERT from scratch including the newline token, it may suffer
i can't think of a solution besides finding a model out there that doesn't do away with newlines (treat it as whitespace)

the code will also have to be modularized to accept different models and their encoders, as a lot of the logic is specific to BERT-base

Notebook When training RETRO with the standard methods, TrainingWrapper does not add line breaks to the dataset. This can have a bad effect on many NLP tasks.
Input *.txt:
First Citizen:
We are accounted poor citizens, the patricians good.
What authority surfeits on would relieve us: if they
would yield us but the superfluity, while it were
wholesome, we might guess they relieved us humanely;
but they think we are too dear: the leanness that
afflicts us, the object of our misery, is as an
inventory to particularise their abundance; our
sufferance is a gain to them Let us revenge this with
our pikes, ere we become rakes: for the gods know I
speak this in hunger for bread, not in thirst for revenge.

Second Citizen:
Would you proceed especially against Caius Marcius?

All:
Against him first: he's a very dog to the commonalty.

Model output after traing:
some - - on my head, were even so salts to death strike That which may bet with tears I have found to life, which sweeter than now to dony : be known betwixcombed oaths ring yet in Corioli turnseth from him Dear life redeems doth thinkment for faith ; Or shall be slack than death within this face, PETRUCHIO : Now, wind and house or free thee better now. KATHARINA : Now, in mine honourable fellow : in your chat with me to be it, alive, I think, If to use than my wife, if this rebellious earth Have you will break out The strange s of yours cro

@0x7o Notebook is not accessible (says link don't exist). Can you please share working link. Its very important for me. Thanks

@0x7o would you share your notebook from above? If not, that's cool, or if its long gone, get that.
Thank you,
-steve

I appreciate your interest immensely, but I no longer have access to this notebook as it has been irretrievably deleted.


0

0

@BctfN0HUK7Yg
Unfortunately, I have a lot of work, for now, so the project is not maintained for now. I really want to continue it, but now I have too much work with other projects.
I've old binaries for Windows, so I can share them if you want to.


Change backend directory structure naming from 'source' to 'directory'

This is an effort to help the slightly confusing naming we have in mcweb.
This will involve numerous changes all across the app:

Django:

settings.py
urls.py
'sources'  -> 'directory'
ensure all imports are correct

RTK/Frontend:

will need to adjust api calls in sourceApi.js

request: if you plan to rename /api/sources to /api/directory, please consider briefly having both available (two path lines in urls.py), so that rss-fetcher doesn't have to make the change at the exact same time!


fix: engineering has agreed to keep the z_getnewaddress RPC endpoint.

Reason - z-to-z transactions would still be feasible after its removal (z_sendmany change is always returned to from-address); so its removal would just make things harder for the z-features but does not prevent importing externally generated keys/addresses.

@cronicc Would like your review / approval as well.  Feel free to merge when ready.


Added a help command

This pull request will add a help command to the bot. This will implement the feature resqeted in-, and close #32.
The command allows the user to list every command or get the specific usage of a specific command. The help messages are sent as private messages, to minimise clutter in the chat

Furthermore, if a command is misspelled, when asking for a specific command, the bot will suggest the most plausible match of the misspelled command, using Levenshtein Distance.

Hmm seems it won't pass the ci as it expects tab instead of spaces. I'll correct this in a commit coming soon

You should be able to run the linter, might help ya out! Thanks for the
help Zaph :D
Le sam. 12 oct. 2019 à 18:33, Zaph<EMAIL_ADDRESS>a écrit :

Hmm seems it won't pass the ci as it expects tab instead of spaces. I'll
correct this in a commit coming soon
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Knights-Of-Academia/horace/pull/33?email_source=notifications&email_token=ADATWBWJQRBWUBCI2CVPJ63QOJGDZA5CNFSM4JADX5OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBCJTXA#issuecomment-541366748,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADATWBS76K64I2KHPSTQJJDQOJGDZANCNFSM4JADX5OA
.

Hmm seems it won't pass the ci as it expects tab instead of spaces. I'll correct this in a commit coming soon

Thanks for your contributions so far! There's an npm script called fixlinting which should fix it without you having to do much.
Your solution looks elegant, nice idea to include Levenshtein for unknown commands. 👍

All tests now pass in the CI. Should be done and ready to implement

That should be corrected now :)


electron - angular 2 - windows

I am new in angular and electron.
I have problem with choosing db for my new project.
The project will be offline application on windows. I will use electron and Angular 2.
Please help me, what DB should I use? I cant find DB with is suitable for windows and angular 2

I am trying the same... I want to use this, I hope it will run... I managed to somehow create table in memory and read it by the moment... Installation was biggest struggle, also packaging is something I need to finish right, it only works with electron-packager, not electron-builder.


App asking for biometric while it's not enabled

How to reproduce on Android:

Install and launch the app while having biometric enabled
Disable biometric from the phone
The app will now ask for biometric every time and I have no way to disable that from the app.

It's a bug because the app should check everytime if biometric is enabled or not.
Testing
Test the following cases on Android:

[ ] Device does not have biometric hardware -> should show PIN code field only
[ ] Device has biometric but it's not enabled -> should show PIN code field only, no error dialog prompting user to enable biometric
[ ] Device has biometric enabled when user signs up, later disables it -> should show PIN code field only,  no error messages
[ ] Device has biometric and it is enabled -> should accept biometric in lieu of PIN

Perhaps a stupid question, but how do you disable biometrics? I have a Pixel XL with Android 10 and there only seems to be the option to remove fingerprints. I can't find any way to totally disable the feature on the phone.

You don't really disable it but instead, remove all registered fingerprint.
I tested, registering a fingerprint because I'm not using that usually. After that every app propose me fingerprint auth (with backup to pin code if you want). After I remove the fingerprint, everything related to fingerprint disappeared except in Seeds.

Ah I see! Thanks for noticing that 👍

I got a weird message when installing it on the android simulator for the first time - might be the same bug.
I entered a pin code, then it asked me for a fingerprint . Hmm.. but it recovered.

it seems like the lib that checks if fingerprints is available always return true n matters if you have a fingerprint added in your settings or not. Might be a tricky one ....

Well I guess we're not the only one https://github.com/flutter/flutter/issues/45497 ....

^^^ good find! That ticket contains a workaround for fingerprint... applicable?

Yeah that lib is missing at least two use cases. We want to know if the user has a fingerprint or face added, not if the device supports it as you both have mentioned. And also we want to be able to detect if the fingerprint/face has changed to know if we should take action like require the private key again (like you mentioned you've seen in other wallet apps Nik).

The alternative lib looks promising. Have you tested it in any way?

The alternative lib looks promising. Have you tested it in any way?

No. @divyanshub024 is going to take a look

@gguijarro-c-chwy did this it's been merged


State of VulkanSharp

Currently it seems that there is no further development or bugfixing for VulkanSharp. Issues are not commented for some time now. The generator does not work for current vk.xml.
That's why I wanted to ask: What are the plans for VulkanSharp? Will there be any further work on it or should I switch to one of the other vulkan wrappers?
It's not that I want to force anyone to work on VulkanSharp and I totally understand that open source projects are lots of work. I really appreciate all the work that has been put into this project. I just need to know which project is most promising for the future.

The next thing I plan to work on is Vulkan 1.1 support, so that we have it available for Xamarin Android.
I would like to have it ready in about a month. Depends how much time I can squeeze in between the Xamarin Android work.

Great to hear that VulkanSharp is under active development. Don't feel pushed by me. Just wanted to know if I have to look for alternatives on the long run. But now I am happy to know I don't have to invest that time. Thanks!

Hello Radek, yust a few days ago I have committed an update to Vulkan 1.1.87 in my fork. Maybe you want to use it. This is my fork: https://github.com/MaikBusch78/VulkanSharp


hi ! how do i run lightweight charts on nodejs ?

i have nodejs v4.2.6 on ubuntu linux....
i don't know how to start running the application....
your help is much appreciated !
tal

okay - i saw lightweight charts require nodejs version at least v10....
i have the files for linux - of nodejs v12...
but i don't know how to install it...

if i try to update or upgrade the nodejs version through the terminal
it gives me this towards the end:
Errors were encountered while processing:
linux-firmware
initramfs-tools
linux-image-4.4.0-150-generic
linux-image-4.4.0-154-generic
E: Sub-process /usr/bin/dpkg returned an error code (1)

Hi, lightweight-charts is client-side library, it could be used in the browser. I don't know whether it's possible to use it in nodejs. nodejs version in package.json in min version you could build the library and run tests for it.

Why do you need to run it in nodejs? What is your aim?

well, how do i run it in the browser ?
if i try to 'throw' files into the browser (like index.js for example) - it just shows the code...

I think it's better to ask such question on stackoverflow - it's doen't related to the lightweight-charts itself, you can ask such question related to anything.
https://stackoverflow.com/questions/16333790/node-js-quick-file-server-static-files-over-http

If you build the library locally, you can use suggestion from https://github.com/tradingview/lightweight-charts/blob/master/BUILDING.md#tips:

If you want to play with a locally built package, you can create a debug.html page: cp debug.html.example debug.html.


Refactor Store class

What, Why
To handle the online token #9 we need to rethink how we are doing the whole Store class.
At the same time let's improve the API with an overall rewrite.
How

StoreAPI or StoreABC (Naming is hard) abstract class with methods that need to be overloaded:

save_online_token and save_offline_token to be called at the end of the OAuth process to store in DB, or wherever needed/wanted
load_online_token and load_offline_token to be called whenever the load method initiates a store

load function to instantiate the Store class with an async call:
* If argument is just store name (store subdomain) then load offline token: store = await Store.load(storename=storename)
* If argument is the store name and the staff id then load online token: store = await Store.load(storename=storename, staff_id=staff_id)
Rename shoprequest to something execute_rest

Could we do something like the following?
 Application
  |-- Store[]
  |      |--  OnlineToken[]
  |      `--  OfflineToken
  |-- Auth

Application (Class): A container for all of the stores and the authentication. This allows us to remove the instance of a store that holds a store logic that currently exists making it clearer the scope and functionality of a store. It allows us to share one object between all routes giving access directly to all store instances and authentication endpoints without having to query for information. For example, instead of having to do the following:
dbstore = await DBStore.find_one_by_name(name=storename)

return Store.load(store_id=storename, name=storename, access_token=dbstore.access_token)

We can instead something like the following:
application.getStore(storename)

Which would return an instance of a store loaded in our application.
Store (Class): A store contains the core functionality for making queries on a single store, it holds its own authentication tokens, both online (up to n) and offline (up to 1).
Auth (Class): This handles all of the logic associated with oauth. Instead of initializing the routes separately from the original application, this combines the OAuth portion into the initialization of the application so you only have to make one call.
my_application = Application(
    app_domain,
    app_scopes,
    client_id,
    client_secret,
    shopify_handle,
    user_scopes,
    callback_path,
    install_init_path,
    [stores]
)

Token (Abstract Class): This token abstract class contains most of the logic for a token, providing the functionality to create and renew a token. This should not be instantiated directly. This lets the programmer implement their own logic for how the save and load functionality for a token works (if necessary).
OnlineToken (Class): This provides the functionality for an online token (information about user). We can set this up to be a default approach for storing an online token.
OfflineToken (Class): This provides the functionality for the basic offline token. We can set this up to be a default approach for storing an offline token.

So I'm gonna do what I should have done earlier (My bad! :sweat: ) and document clearly what I have in mind and a little why this approach. Slack is the worst to agree on design!  :expressionless:
The code below my have errors obviously. It's for design reference.
We just need 3 classes to rewrite the current Store class (which handles only offline tokens). I believe we can make them all as BaseModel from pydantic but that might not work.

First a TokenABC with all the methods and attributes shared by online and offline tokens. At this time it's best to pretty much copy paste most of the Store class, as much as possible. I put here just the two main method to more or less copy paste.

class TokenABC(BaseModel, ABC):
    access_token: str
    store_name: str
    <and the other attributes>

    async def execute_rest(<stuff>):

    async def execute_gql(<stuff>):

The OfflineTokenABC class which will also be an abstract class. It should pretty much be TokenABC in that case.
FYI there is a way to make the return type for load to be the inherited class and not OfflineTokenABC. Use Generic

class OfflineTokenABC(TokenABC):
    @abstractmethod
    async def save(self):
        pass
    @abstractmethod
    @classmethod
    async def load_token(cls, store_name: str):
         pass

The OnlineTokenABC almost the same as above:

class OnlineTokenABC(TokenABC):
    staff_id: str                         # Might as well make it a class attribute. We don't have to though ...
    @abstractmethod
    async def save(self):
        pass
    @abstractmethod
    @classmethod
    async def load_token(cls, store_name: str, staff_id: str):
         pass

Now in terms of usage, using the OfflineToken as example:
class OfflineToken(OfflineTokenABC):
     async def save(self):
        <Implement saving the class to the DB>
    @classmethod
    async def load(cls, store_name: str, staff_id: str):
        <Implement loading the class from the DB and create the OfflineToken instance>

# Then they use it through out their code:
offlinetoken = await OfflineToken.load(store_name='mytshirtsstore')
await offlinetoken.execute_rest(<stuff>)

Between the abstract class and pydantic, it should be really hard for the user to implement an error!
We'll implement a method for dependency injection as suggested by Quinn later on.
Let me know if you have any question.


AD0001 null reference exception

@pjanotti @terrajobst
Using 0.2.15-alpha:
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
8>CSC : warning AD0001: Analyzer 'Microsoft.DotNet.Analyzers.Compatibility.Deprecated.DeprecatedAnalyzer' threw an exception of type 'System.NullReferenceException' with message 'Object reference not set to an instance of an object.'.
And there is much more of them!
Same with 0.1.3
Any way to provide you additional info to fix this issue?

@Krakean is it possible to share the project or solution? I have a small set of test projects and I am not hitting it... it seems that some pattern in the source code is causing this but I don't have such pattern in my test projects. If not possible I will give a try at some random projects, but it is more timing consuming that way. BTW, thanks for the detailed report.

@pjanotti
No problem. Give me your email please (mine is open, you can see it in my github profile, just in case if you wont share you publically), so I'll send you a link.

@Krakean I just sent you an e-mail (my e-mail is public too).

Using 0.2.15-alpha:

I'm confused by this version number. We only have a 0.1.2-alpha and 0.1.3-alpha. Which version are you using?

@pjanotti Sent.
@terrajobst
Just added to nuget sources this one - https://i.imgur.com/bdXWdKo.png - to get latest binary build.

@pjanotti Heh: https://i.imgur.com/MZEJsij.png
My email was contain .zip archive. It wasn't big, something around ~5mb.
Resend with a link to archive instead of attachment. Check email.

@pjanotti have you received a link to the project?

@Krakean of course they went straight to junk mail... but now I got them, I may only be able to look at them at the beginning of next week.

Thanks @Krakean! The issue was very simple: the code was assuming that in certain cases there was always a containing type or method which is not true for pointers - I'll add some unsafe code to my projects with some pointer ops.

@pjanotti
Thanks. Can you please update Nuget with fixes? :)

Yeah, sorry about that... it takes a bit of extra effort to get one published because it needs to be signed. I will get the process started, I will let you know when it becomes available.

@terrajobst
Have you fixed your weird versioning of nightly builds at https://ci.appveyor.com/nuget/platform-compat ?

@Krakean the nugget 0.2.12-alpha was released earlier today.


[Snapshot testing] className mismatch depending on all or single test run

[X] I have searched the issues of this repository and believe that this is not a duplicate.

Update: It's reverse, with all tests run the class is MuiSelect-root-146 and with one test run it's MuiSelect-root-121.
Are you familiar with such case? What do you recommend to prevent this?
Steps to Reproduce (for bugs)
At the moment it's just a question. If it will become too annoying I will create a sandbox.
Your Environment

Tech
Version

Material-UI
1.0.0-beta.45

React
16.3.2

@arteniioleg It's a duplicate of #9492. If the issue isn't enough, we can think of adding a guide in the docs. Let me know.

Add in docs please. In the issue there are many code examples, I would like the recommended one.


fix(DDIDS-1023): Removing border radius from notification banner

Removed border radius from the banner.
Few files were formatted.

:tada: This PR is included in version 4.0.0-alpha.84 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.125 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.18 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0-alpha.11 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Use client preset with imported types

Is your feature request related to a problem? Please describe.
Currently the typescript-operations plugin allows me to use the import-types preset and with preResolvesTypes to generate operation types which pick from existing types (generated elsewhere). I would like to have this same behavior when using the client preset.
Describe the solution you'd like
I would like to be able to specify preResolveTypes: false and provide a typesPath (same as currently available for import-types preset) when using the client preset. When set the client preset should not generate new top level operation types but should instead pick from the imported types from the typesPath (again same as is currently setup with import-types)
Describe alternatives you've considered
I have tried to use a combination of existing plugins to get the same behavior as client but with imported types. This has almost worked but the missing piece is generating the gql function which the client function is doing with the currently undocumented gql-tag-operations (https://github.com/dotansimha/graphql-code-generator/blob/master/packages/plugins/typescript/gql-tag-operations/src/index.ts).
My current config has the following components which successfully generates the types (with a custom plugin on the original schema types).
    './src/__gql__/schemaTypes.ts': {
        plugins: {
            './graphql/plugins/myPluginThing.js': {},
            'typescript': {
                nonOptionalTypename: true,
            },
            'typescript-operations': {
                skipTypename: true,
                dedupeFragments: true,
                preResolveTypes: false,
            },
        },
    },
    './src/__gql__/operationTypes.ts': {
        preset: 'import-types',
        presetConfig: {
            typesPath: './schemaTypes',
        },
        plugins: {
            'typescript-operations': {
                skipTypename: true,
                dedupeFragments: true,
                preResolveTypes: false,
            },
        },
    },
    './src/__gql__/apolloClient.ts': {
        preset: 'import-types',
        presetConfig: {
            typesPath: './operationTypes',
        },
        plugins: {
            'typescript-react-apollo': {
                withHooks: false,
            }
        },
     },
    './src/__gql__/typedDocNode.ts': {
        plugins: {
            'typed-document-node': {}
        },
     },

This all works great but we currently rely on the types generated by gql-tag-operations in the client preset to types the gql queries we write e.g. gql(query { stuff { id } }) has a return type of a TypedDocumentNode (rather than just the document node when using apollos gql function).
I considered using this gql-tag-operations plugin but was unsure how to generate the SourceWithOperations object which is handled here in the client preset: https://github.com/dotansimha/graphql-code-generator/blob/master/packages/presets/client/src/process-sources.ts.
Is your feature request related to a problem? Please describe.
I think the most ideal solution to me is client is extended to support importing types. Alternative better documentation on the gql-tag-operations might make sense as it might be a simple to plugin that in? Equally I could just be missing a simple fix for what I'm trying to do so any help if much appreciated.

Also in the documentation it currently states you can provide a list of presets

generates.preset - A list of presets to use for the output. Presets are a way to dynamically create the list of output files based on the input schema. near-operation-file-preset is a good example

Is that correct? I haven't seen any examples of this and couldn't understand how this would work but if that's the case it might be ideal?


Improved Valora Invite Experience

Expected Behavior
TODO: Add Figma cc @coreycelo
Current Behavior
What is the current behavior?

Designs: Figma


RACCommand's executionSignals's subscribeNext block or error's block execute more times?

i use RACCommand to send a network request in a sendController, you can click this controller right barbuttomitem, then will execute a command; if success, this sendController will be dismissed;
but when fail, you know i add the log in the errors's subscribeNext block; i click barbuttomitem first time, the log will be show one time; but when i click at the second time, the log will be show two times; and it will always added one times if i continue click;
so, i want to know is there some solution to settle this problem? thanks!

You only need to call the error's subscribeNext: once. It sounds like you're calling it multiple times. Can you share some code?

sure; the view like below picture;

if the server send error like 403(cause my app don't be verified so the server limit the interface call times). And the user can did this sendBtn more times, so the errors subscribeBlock can be executed more times, as same as the command be executed more times too; Here is my code, so can i have some solution to avoid this kind of problem?
[[_sendBtn rac_signalForControlEvents:UIControlEventTouchDown] subscribeNext:^(id x) {

            [weakSelf.sendViewModel.sendCommand.executionSignals.switchToLatest subscribeNext:^(id x) {
                MBProgressHUD *hud = [MBProgressHUD showHUDAddedTo:self.view animated:YES];
                hud.mode = MBProgressHUDModeText;
                hud.labelText = @"已发送";
                [hud hide:YES afterDelay:1.0];

                dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(1 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
                    [self.view endEditing:YES];
                    [self dismissViewControllerAnimated:YES completion:nil];
                });
            }];

            [weakSelf.sendViewModel.sendCommand.errors subscribeNext:^(id x) {
                LJLog(@"()()()()"); // click more times the text will be loged more times
                MBProgressHUD *hud = [MBProgressHUD showHUDAddedTo:self.view animated:YES];
                hud.mode = MBProgressHUDModeText;
                hud.labelText = @"新浪服务器开小差了...";
                [hud hide:YES afterDelay:1.0];
            }];

            NSDictionary *dict = @{@"statusd":_textView.text};
            [self.sendViewModel.sendCommand execute:dict];
        }];

You shouldn't subscribe to errors or executionSignals every time the button is pressed. It should only happen once.
            [weakSelf.sendViewModel.sendCommand.executionSignals.switchToLatest subscribeNext:^(id x) {
                MBProgressHUD *hud = [MBProgressHUD showHUDAddedTo:self.view animated:YES];
                hud.mode = MBProgressHUDModeText;
                hud.labelText = @"已发送";
                [hud hide:YES afterDelay:1.0];

                dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(1 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
                    [self.view endEditing:YES];
                    [self dismissViewControllerAnimated:YES completion:nil];
                });
            }];

            [weakSelf.sendViewModel.sendCommand.errors subscribeNext:^(id x) {
                LJLog(@"()()()()"); // click more times the text will be loged more times
                MBProgressHUD *hud = [MBProgressHUD showHUDAddedTo:self.view animated:YES];
                hud.mode = MBProgressHUDModeText;
                hud.labelText = @"新浪服务器开小差了...";
                [hud hide:YES afterDelay:1.0];
            }];

[[_sendBtn rac_signalForControlEvents:UIControlEventTouchDown] subscribeNext:^(id x) {
            NSDictionary *dict = @{@"statusd":_textView.text};
            [self.sendViewModel.sendCommand execute:dict];
        }];

ok, thank you very much.


Bulma released v0.9.0

Please take action to support Bulma v0.9.0 in @nuxt/bulma. This version has some really useful updates.

I support this! I would like to use some of the new functionnalities from Bulma 0.9.0!

Sorry for the late review. Merged and released<EMAIL_ADDRESS>


Fluent validator makes 500 ErrorCode

I have created a .net core solution and bound FluentValidator to the project by manual. And for example this rule:
 public class CommandValidator : AbstractValidator<Command>
        {
            public CommandValidator()
            {
                RuleFor(x => x.ValueName).MinimumLength(2);
            }
        }

works pretty nice ! y controller returns Error 400 and proper message.
However, I have create another one:
            public CommandValidator(ITournamentDal tournamentDal)
            {
                RuleFor(x => tournamentDal.Get(x.TournamentId))
                    .NotNull();
            }

My controller returns 500 message with ugly response payload. I should note that tournamentDal.Get(x.TournamentId) works properly and returns Null.
And... I am just get lost. Please help.
P.S. "
FluentValidation.ValidationException: Validation failed:
-- '' must not be empty.
…..
…...
"

Hi, please provide the following details:

Which version of FluentValidation are you using?
How are you invoking the validator? (Please provide the relevant code)

It sounds like you’re using ValidateAndThrow to throw an exception if the rule fails. If this is a web project then you shouldn’t be using ValidateAndThrow, you should be putting the errors into ModelState.
Also, Rules should be against properties, but you’re calling a method inside your RuleFor. You should rewrite the rule to be directly against the TourtnamentId property, eg:
RuleFor(x => x.TournamentId).Must(id => tournamentDal.Get(id) != null).WithMessage(“Please specify a valid id”);

RuleFor(x => x.TournamentId).Must(id => tournamentDal.Get(id) != null).WithMessage(“Please specify a valid id”);
Unfortunately, it works absolutely same.
I am using FluentValidation v7.6.105
in ASP.net core project. With Mediatr library.
`    public class Close
{
public class Command : IRequest
{
public Command(int tournamentId)
{
TournamentId = tournamentId;
}
        public int TournamentId { get; }
    }

    public class CommandHandler : IRequestHandler<Command>
    {
        private readonly ITournamentDal _tournamentDal;

        public CommandHandler(ITournamentDal tournamentDal)
        {
            _tournamentDal = tournamentDal;
        }

        public async Task<Unit> Handle(Command request, CancellationToken cancellationToken)
        {
            await _tournamentDal.UpdateTournamentClose(request.TournamentId);
            return Unit.Value;
        }
    }

    public class CommandValidator : AbstractValidator<Command>
    {
        public CommandValidator(ITournamentDal tournamentDal)
        {
            RuleFor(x => x.TournamentId).Must(id => tournamentDal.Get(id) != null).WithMessage("Please specify a valid id");
        }
    }
}`

Hi, I can't help you with Mediatr I'm afraid, only FluentValidation, but if you're seeing that exception it means you're calling ValidateAndThrow to invoke the validator, rather than Validate. If the exception is being thrown then it means that the rule is failing (and the call to tournamentDal.Get(id) is returning null.
(also just a tip, formatting code blocks in github is done by 3 backticks).

Mediatr is OK. Nothing special. Simply as you can see above, one Validation works and other not.
The main difference I found is constructor by default.

I'm still having trouble understanding what's not working....as I said before, if you're seeing an error it means the rule is failing. What exactly isn't working for you?

PLease view sample here:
https://github.com/OlekRia/Test125
And especially this one: https://github.com/OlekRia/Test125/blob/master/Duplicate.BJ.Application/Features/Tournaments/Close.cs
makes 500 error
but
https://github.com/OlekRia/Test125/blob/master/Duplicate.BJ.Application/Features/Values/Create.cs
makes 400 response with pretty payload as usual for FluentValidator

The problem is your ValidationPipelineBehavior class:
https://github.com/OlekRia/Test125/blob/16bf468894b57e891a0e749944fb219ec6422d13/Duplicate.BJ.Bootstrapper/ValidationPipelineBehavior.cs#L19-L34
As I said previously, you shouldn't be throwning an exception - you should be adding the validation results to ModelState.
Throwing an exception which is unhandled will cause ASP.NET to generate a 500 response. This isn't related to FluentValidation, that's just the standard behaviour of ASP.net when dealing with unhandled exceptions. If you add the results to ModelState, then a 400 response will be correctly generated. Your ValidationActionFilter will also never be called as the exception has already caused the request to fail.
The other command (https://github.com/OlekRia/Test125/blob/master/Duplicate.BJ.Application/Features/Values/Create.cs) does exactly the same thing, because of the ValidationPipelineBehaviour is throwing that exception. The other Create command (https://github.com/OlekRia/Test125/blob/16bf468894b57e891a0e749944fb219ec6422d13/Duplicate.BJ.Application/Features/Tournaments/Create.cs) does run, but that's only because the validator is not implemented, so is never being invoked.
I can see a couple of options:

Find a way to inject the current ControllerContext into your ValidationPipelineBehavior and then copy the validtion results into the ModelState dictionary in here
Keep the ValidationPipelineBehavior as-is, but use an exception filter higher up the chain to catch the ValidationException, extract the failures from the exception add then results to ModelState inside the filter.

I am going to close this issue now as the problem is with your architecture rather than a problem with FluentValidation itself.

So this: https://stackoverflow.com/questions/42283011/add-validation-to-a-mediatr-behavior-pipeline
answer obviously is wrong!?

Correct; by itself that isn’t a good answer. There needs to be a way to catch/handle the exception to copy the results to modelstate.

The simplest way to make it work without if you don’t want to change the infrastructure would be to wrap the call to mediatr in your controller action with a try/catch that handles the ValidationException, extracts the failures and copies them to modelstate. Not as elegant admin using a filter, but at least it’s skmple.

I will try. Not easy to understand, however. Simply Mediatr uses this sample as official feature (using validation in pipelining) in the way I just described above.
Anyway, it's hard to understand why Value Create is OK and Tournament Create is not. They seem really absolutely same !

I tried your application and ValueCreate actually does exactly the same thing if validation fails.
I’m out at the moment but will try and post an example later when I’m home to try and make it easier to understand.

OK. Howerver, it's really hard to understand the difference between two Creates.
When I go to ValueCreate I don't even go to pipeline in Mediatr.

This is what I meant by handling the exception and copying the messages to ModelState:
[HttpPut("{tournamentId}")]
[ProducesResponseType(204)]
public async Task<IActionResult> Close(int tournamentId)
{
    try
    {
        await _mediator.Send(new Close.Command(tournamentId));
    }
    catch (ValidationException ex)
    {
        CopyValidationErrorsToModelstate(ex);
        return this.ValidationProblem();
    }

    return NoContent();
}

private void CopyValidationErrorsToModelstate(ValidationException validationException)
{
    foreach (var error in validationException.Errors)
    {
        ModelState.AddModelError(error.PropertyName, error.ErrorMessage);
    }
}

Your approach is good, but would be better if we don't make this error check for each controller action. Guess it could be nice to pass somehow model state to pipeline somehow.