[NLPCRAFT-394] and more - Create a virtual/conda environment for python inside a new .nlpcraft-python folder

This pull request should achieve the desired objective of the following JIRA tickets:

NLPCRAFT-394
NLPCRAFT-411
NLPCRAFT-413
NLPCRAFT-393
NLPCRAFT-186

NLPCraft will now require conda > 4.8 for the Python section to run.
This change does the following:

Creates a <USER_HOME>/.nlpcraft-python folder, where python setup and dependencies are managed. This is done in the pom.xml of the nlpcraft module.
Creates a conda environment within the .nlpcraft-python folder so that system python versions and dependencies are completely untouched. This will also be much better for a production type of architecture.
Uses conda as a requirement to set the Python version that we can use for the project. Currently, I have set the Python version we will support for NLPCraft , to Python version 3.8 . This is also in the pom.xml
python_setup.py file created that manages the installation of fasttext, pytorch and the other python dependencies in an OS independent way.
Fixed the start_server.sh script to start the ctxserver and made it location independent. That is, it can be started from anywhere.
Added TODOs
The installation is largely done during the Maven build and partly by a Python script that I have tried to make OS independent
Updated some Readme files
Instead of multiple Flask servers, I have set it to one, with REST naming and different endpoints as blueprints. This will make adding additional things like tokenProviders as simple as creating another blueprint and adding it
Setup python to be able to read settings from the nlpcraft.conf file and function accordingly.
Removed OS dependent download scripts and dependency installation scripts.
Made the BERT model download into the ctxword/bertft/data folder OS independent by including it in the Python script bertft.py. Initial startup will be slow but once the file is available on your hard drive, it will be faster the next time.

CTXServer Note: Currently, the ctxserver will need to be started manually from the start_server.sh file. Starting it automatically is another JIRA ticket (NLPCRAFT-373) that I will work on as well.
Build Note : mvn clean package verify works for me on Ubuntu 20.04 and Mac OS Big Sur. I have not tried Windows. I dont have a development system with it yet.
Pom.xml

Validate stage contains the verification for the conda version, creates the .nlpcraft-python folder
Compile stage creates the conda environment. It also contains the installation of pytorch, fasttext and other python dependencies.

To do further:

Make the conda verification better by more accurately checking it
Fix the windows ctx server start script
More documentation needed
Package the json files in the .cache folder used by the HuggingFace transformers dependency and make this work offline

Hi!
I have few questions for discussion:

I don't think that we have to install python stuff from maven command. I guess it is not standard way.
I think it should be scripts - install.sh(cmd), maybe with uninstall.sh(cmd)

Do we need conda ?  (I am not familiar with python tools, is it necessary for python?)
I ask this question because we require too many tools for nlpcraft installation.

java
maven
pytnon 3.8/3.9
and conda now

If we need conda I guess we have to require user to configure CONDA_HOME env variable, but shouldn't ask him to update scripts to set path to conda.

Where is bert models etc installation place?

Now it is <project_root>/nlpcraft/src/main/python/ctxword/
Is new place under <user_home>/.nlpcraft-python ?

OT - if yes - we have to delete from .gitignore **/__pycache__, **/data and **/.ipynb_checkpoints

When the models should be downloaded?

during installation (I think so) or
when first attempt to start server?

Related questions

I guess we need to add some progressbar for such long processes as downloading models (It is not clear  - is the process going and hanging)
When downloading process is killed on some intermediate phase (for example, some files downloaded, some aren't, some broken etc) - how should it be resolved?
Is it supported to fix such situations?
I guess server should be started and accept HTTP connections after all models downloading process finished.

Problems:

When I deleted models from <project_root>/nlpcraft/src/main/python/ctxword/, start-server script printed that downloading in progress, but new models files weren't appear.
The process hangs and I don't see any errors.

Other minor issues

org.apache.nlpcraft.server.nlp.core.spacy.NCSpaCyNerEnricher.Config - should be fixed
Why some logs seem duplicated?

Below log for one start
2021-08-16 11:32:02,783 - bertft - WARNING - CUDA is not available for Torch.
2021-08-16 11:32:02,784 - bertft - INFO - Initializing fast text
2021-08-16 11:32:02,784 - bertft - INFO - Found existing model, loading.
2021-08-16 11:32:02,871 - bertft - WARNING - CUDA is not available for Torch.
2021-08-16 11:32:02,872 - bertft - INFO - Initializing fast text
2021-08-16 11:32:02,872 - bertft - INFO - Found existing model, loading.
11:33
….
2021-08-16 11:32:40,614 - bertft - INFO - Server started in 37.8301 seconds
2021-08-16 11:32:40,639 - transformers.modeling_utils - INFO - Weights of RobertaForMaskedLM not initialized from pretrained model: [‘lm_head.decoder.bias’]
2021-08-16 11:32:40,641 - bertft - INFO - Server started in 37.7691 seconds
2021-08-16 11:32:41,903 - root - DEBUG - Registering ctxserver blueprint
2021-08-16 11:32:41,903 - root - DEBUG - Registering ctxserver blueprint
11:37

Hello,
Thank you for reviewing this thoroughly.
To answer your questions (I'm sorry for this being lengthy but I don't know how else to convey this):

I don't think that we have to install python stuff from maven command. I guess it is not standard way.
I think it should be scripts - install.sh(cmd), maybe with uninstall.sh(cmd)

If the Ctxserver is considered to be an integral part of NLPCraft, I would disagree with this for the following reasons:

Verification of every release will become more complex. This is what is currently happening where mvn clean package verify (or its variants) are being used as a factor to determine if the release is good without checking the Python part. It will be much cleaner and more accurate to validate a release if there is a single way to build this that tests all the areas of the project and their dependencies. The setup scripts running in Python with standard libraries (given we require Python > 3)  and making it OS independent helps, in my opinion.

In my opinion, implementation of Maven builds are project dependent. In cases where a project has dependencies on multiple languages (and their dependencies). I think this way (pull request 18) is a good way to do it, considering that our project falls in this area. I think it will be more beneficial if we build profiles per operating system. In my personal experience, I have seen this implementation used in a production system.

One of the issues with .sh and .cmd scripts is that they are OS dependent. Sure, we can build profiles and then implement the script based on the OS. That is also a solution but a different one.

Do we need conda ? (I am not familiar with python tools, is it necessary for python?)
I ask this question because we require too many tools for nlpcraft installation.

Managing Python and its dependencies is not as streamlined as Java is (in my opinion).
Regarding Python, most users today, download Anaconda or Miniconda and install Python that way. Conda is a package management and environment management system for Python. It comes with Anaconda and Miniconda and is extremely common in the Python world.
Case example (for why we need a virtual environment for Python):
Suppose your system python version is 3.5. You use dependency X with version 1.1. Now if you want to install NLPCraft and NLPCraft needs dependency X with version 1.3. Installing NLPCraft will uninstall 1.1 and you will have an issue with anything else that has used 1.1 (assuming something that you are using, changed between 1.1. and 1.3).
Also, certain dependencies support certain Python versions at times. For example,  there was a case when Tensorflow supported Python 3.6 and Python 3.7+ was released, this is one such example.
@skhdl Note: I'm just explaining in case someone else who is not as familiar with the project and dependency management reads this. I'm well aware that you are very familiar with dependency management and I don't mean to be condescending :)
The production like way to do this (without taking Docker as a factor), in my opinion is, to have a virtual environment so that the system Python is not affected. In this case, you are left with largely two options:

virtualenv
conda

The issue with virtualenv is that it uses only the system version of Python which the user has. For example, if you have 3.5, your virtual environment will use Python 3.5 only.
Conda lets you manage the version of Python as well. This is primarily the reason I chose it. In this case, for our project, we eliminate technical debt by supporting one specific version of Python for this project. We will not have to care if 3.9, 4.0 etc. gets released, unless our dependencies are affected. We also give the developer the option to set the Python version and environment (in the way I currently designed it).
In my opinion, a conda environment is a good option in production as well. All the developer has to do is either create a environment.yml file to recreate the conda environment or just make the entire folder a zip file and install it that way. I work with the assumption that, in production, I need to assume that there is no internet connection available after packaging. This is the case in many companies (especially Financial ones). My goal was to take our project as close to a real world production implementation as possible so there is minimal work left for the developer using our project.

If we need conda I guess we have to require user to configure CONDA_HOME env variable, but shouldn't ask him to update scripts to set path to conda.

I agree. I could work on this.

Where is bert models etc installation place?

Same location. I did not change this. <project_root>/nlpcraft/src/main/python/ctxword/
Eventually, we should also put this in .nlpcraft-python

When the models should be downloaded?

during installation (I think so) or

I think so too. This pull request is to first establish a virtual environment for Python and get everything setup there. We can deal with the model download location once that is complete. This is my opinion.

I guess we need to add some progressbar for such long processes as downloading models (It is not clear - is the process going and hanging)

Yes, you are right. I can work on that after we establish the python environment separate from the user system python environment. It works fine on Ubuntu 20.04 for me. I tried it on MacOS and you are correct, it hangs. I will work on fixing this.

When downloading process is killed on some intermediate phase (for example, some files downloaded, some aren't, some broken etc) - how should it be resolved?
Is it supported to fix such situations?

Can you give me an example? I believe you might be talking about FastText. I noticed this too recently, and I am fixing it. Thank you for pointing it out.

I guess server should be started and accept HTTP connections after all models downloading process finished.

I agree.

When I deleted models from <project_root>/nlpcraft/src/main/python/ctxword/, start-server script printed that downloading in progress, but new models files weren't appear.
The process hangs and I don't see any errors.

Yes, I mentioned above that I found this issue in MacOS. I'll fix it.

Other minor issues

org.apache.nlpcraft.server.nlp.core.spacy.NCSpaCyNerEnricher.Config - should be fixed

Will look into it and fix it.

Why some logs seem duplicated?

There was a previous issue  which I fixed, where I mistakenly added a second log handler in the logging module. I thought I resolved this. Does it happen even when you pull with the most recent updates?
Note: I also found some issues on Windows that I am working on fixing. I'm not sure if I should keep the pull request open and fix these issues or close it and start a new one. Please feel free to advise me on this.
Thank you for your review.

@rahul3 thank you very much for sharing your excellent work!
@skhdl thank you for detailed review!
Let me also join the discussion of Ctxserver installation.
I think we should keep in mind two use cases:

Development process
End user experience

For us, as developers, it is easier to run Maven commands in order to build and verify changes.
But for end users it can be a tedious task to setup all the necessary tools to build the whole project.
That is one of the reasons of why we have binary releases that users can use to quickly download and run the NLPCarft server out of the box.
In my opinion we should have installation scripts that would help end users to install all the necessary dependencies.
We can also run those scripts via Maven to make developers life easier and to make sure that those scripts get executed as part of our build/verify process.

@rahul3 thank you for your reply. I guess I understand conda role, but still doubt in maven. Will try to investigate.

I'm actually seeing @rahul3 point on maven - maintaining *.sh and *.cmd installation scripts will be a major pain (dev, testing, etc.). So, having a single mvn build that installs/downloads everything (first time) is probably the best way to go. For binary builds - we can package and pre-package everything and anything any way we want...

Agree on conda - major plus in separating outselves from system Python installation.

@syermakov - Thank you for your contribution to this discussion.
We can do both i.e. installation scripts (for the user), and also have the full build in Maven like @aradzinski mentioned. But as @aradzinski correctly (in my opinion), pointed out, it would be a pain and additional technical debt each time we need to verify something etc.
So far, the consensus from what I have read is the following

We all (now) agree on conda.
We are divided on what do about the scripts i.e. the way the installation should proceed.

So for this request, what do you (all) recommend that I work on (apart from fixing all the issues I mentioned in my last comment) for this given task?
My recommendation:
We first put everything to download/install and build in Maven (as it is for this pull request) and like @aradzinski mentioned. And we also open a ticket to create fully functional scripts after this has been implemented. In that way, we could possibly have the best of both worlds.
For me (technical task right now): Currently this pull request won't work on Windows and I'm in the process of trying to fix it. And fixing the other issues that I mentioned above.
I look forward to reading your thoughts on this.

@skhdl  - I have fixed everything except updating the script to start the ctxserver. In my opinion, the script fix to initialize conda depending on the shell, can be part of a different JIRA.
I can confirm that mvn clean package clean verify work on Ubuntu 20.04 and Mac OS Big Sur for me.
We will need to build by using mvn clean package clean verify from now on (because of the requirement to delete the FastText repo after cloning it). Could you please check if the project builds for you?
I'll check on Windows tomorrow and confirm to make sure that the build, is fully functional.

I'll weigh in here.
I think we should start with "everything in maven" approach. It should download/install everything when it detects that something Python related is missing.
@rahul3 I don't like the idea of a non-standard command to build the project. I think we should try to retain the standard "mvn clean package" idiom.

@nivanov

I think we should start with "everything in maven" approach. It should download/install everything when it detects that something Python related is missing.

This is how it is currently built in this pull request.

I don't like the idea of a non-standard command to build the project. I think we should try to retain the standard "mvn clean package" idiom.

To clarify, mvn clean package builds the project even with this pull request.
However, to run the verify phase, you would have to clean it again because of the way FastText is designed to be installed. It is via a git clone <fast-text-repo>. It is somewhat non-standard and I've checked that it has issues cross-platform (Windows specifically) if it is done via pip. This is in the FastText official documentation.
I could also add another folder deletion in the verify phase so that mvn clean package verify would work. Let me know if that is what you would prefer.

I think generally the mode standard the build process - the better. 99% of people never ready any release notes or docs - they just run standard Maven toolchain and expect it work. "Conventions over configuration...."

@nivanov  - Sounds good. The change is complete.
mvn clean package and mvn clean package verify work fine on Mac OS, Linux and Windows for me. Please feel free to check.

Hi!
I didn't review code yet but tried to test it.

I have updated sources
I deleted <user_home>/.nlpcraft-python and all local models which were downloaded into source folder before.
mvn clean package from project root folder

Some remarks:
3.1) We have many suspicious warnings like below, it can confuse users.
src/fasttext.cc:701:26: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int64_t' (aka 'long long') [-Wsign-compare]
for (size_t j = 0; j < dim; j++) {
~ ^ ~~~
3.2) progress bar - very good, but is it possible to decrease progress step ?
.....
[INFO] 931227/4398040K
[INFO] 931243/4398040K
[INFO] 931259/4398040K
.....
We have too many lines in console, it overrides all other output.
maybe better to have just 20 (by 5%) or 50 (by 2%) steps which should be shown? (Or any other way to decrease progress output lines count)

Model downloaded into <user_home>/apache/incubator-nlpcraft/nlpcraft/src/main/python/ctxword/data/cc.en.300.bin
Is it ok? Maybe better to download and use it from <user_home>/.nlpcraft-python ?
Reason - if we download new NLpCraft new version (even minor binary release) we have to install and download everything again.
Maybe better to save these models files in separated folder but not in installation folder?

cd nlpcraft/src/main/python/ctxword
./start_server.sh
ok, it works!

5.1) Have some warnings
./start_server.sh: line 43: deactivate: command not found
IMPORTANT REMINDER: Please do not forget to set the path to your conda.sh file
DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.
5.2) Some duplicated log messages:
BERT ROOT DIR: /Users/skamov/apache/incubator-nlpcraft/nlpcraft/src/main/python/ctxword/bertft/..
BERT ROOT DIR: /Users/skamov/apache/incubator-nlpcraft/nlpcraft/src/main/python/ctxword/bertft/..
CUDA is not available for Torch.
CUDA is not available for Torch.
[2021-08-21 12:09:01]: 64618 DEBUG Registering ctxserver blueprint
[2021-08-21 12:09:01]: 64616 DEBUG Registering ctxserver blueprint

I stopped server

mvn clean package (again)
Seems it doesn't download models again  (rigth? - Do you copy them to <user_home>/apache/incubator-nlpcraft/nlpcraft/src/main/python/ctxword/data from some cache ?
or how do they appear here?)
it starts to download and build some python related staff again. Should it? I guess it should be one time

I deleted <user_home>/apache/incubator-nlpcraft/nlpcraft/src/main/python/ctxword/data folder.

mvn clean package - it creates folder again.. ok.
BTW:
start_server.sh works very slow for me. Previously server started slowly first time, but next starts were quick enough, but now all starts are too slow on my env.

General

I guess we have to provide some detailed instructions and explanations for users related to installation process like

if you first run mvn clean package  - python stuff and models downloaded
second run  - we check that models are already downloaded and do nothing with them - Also I guess python stuff shouldn't be reinstalled. What do  you think ?
if you want to re-download models for some reasons(broken installation etc) - find and delete the folder (which folder) and run mvn clean package  again (like first run)
same for python stuff

Note please that we have some checks: https://github.com/apache/incubator-nlpcraft/pull/18/checks.
Now these tests failed.
We have to support these tests. I guess we have to skip python files installations and all related tests based on some flag.
Something like this nlpcraft/pom.xml line 392

/NCDateGeneratorSpec.*

/NCRestModelSpec1.*

Check configured here <user_home>/apache/incubator-nlpcraft/.github/workflows/build.yml

Hello @skhdl ,
Thank you for the review and the remarks. While all the remarks are certainly valid and need to be worked on, I find that some of  them are out of scope for this pull request. In my opinion, this is a large architectural change and we need to make progress incrementally. I would recommend raising separate tickets for things such as changing the download model location (for one). My replies are below:

3.1) We have many suspicious warnings like below, it can confuse users.
src/fasttext.cc:701:26: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int64_t' (aka 'long long') [-Wsign-compare]
for (size_t j = 0; j < dim; j++) {
~ ^ ~~~

I think that this is a valid concern but is out of scope and out of our control given that we use FastText. This is the result of the FastText installation and this is the gcc compiler throwing warnings.

3.2) progress bar - very good, but is it possible to decrease progress step ?
.....
[INFO] 931227/4398040K
[INFO] 931243/4398040K
[INFO] 931259/4398040K
.....

Downloading is done via a Maven plugin. There are two plugins mainly used from what I have seen (on the internet). There are two options (with the two plugins), either we show the progress in this way (m/n K) or a series of '.......'. This way is also long but I don't think will impact the terminal buffer as much.
Action taken: I've changed it to the other way (.................. +) and used a different plugin (antrun). Let me know if this way is better.
The only other option is that we do not show download progress.

Model downloaded into <project_root>/nlpcraft/src/main/python/ctxword/data/cc.en.300.bin
Is it ok? Maybe better to download and use it from <user_home>/.nlpcraft-python ?
Reason - if we download new NLpCraft new version (even minor binary release) we have to install and download everything again.
Maybe better to save these models files in separated folder but not in installation folder?

I have already mentioned this in my first reply on this thread. I think it is better to download and use from <user_home>/.nlpcraft-python/<some_folder>. This is however, out of scope for this pull request (and for all the JIRA tickets referenced). I would change the way we use Python first and then make these changes incrementally after creating a separate JIRA ticket for the change of model location.

5.1) Have some warnings
./start_server.sh: line 43: deactivate: command not found

This is to deactivate a virtual environment with virtualenv that the user may have created. The thing is, virtualenv is deactivated with deactivate but conda environments are deactivated with conda deactivate. conda used to be also deactivated with deactivate but that has changed and you see a Deprecation Warning. I think we can look into this once we have successfully stepped away from using system Python.

mvn clean package (again)
Seems it doesn't download models again (rigth? - Do you copy them to <project_root>/nlpcraft/src/main/python/ctxword/data from some cache ?
or how do they appear here?)

It will not download models if it already finds them. As you know, the models are large files. The Maven clean phase does not erase the models. I do not think that it is a good idea to delete the models during the clean phase because these large files will need to be downloaded each time.
To your question as to how the models appear there so quickly if you deleted them, the previous plugin I used saved them to a cache. I have changed the plugin (due to the progress bar). Please check now.

it starts to download and build some python related staff again. Should it? I guess it should be one time

Yes, it should. In case there are changes to the Python code and dependencies, the requirements.txt file will change. For regular use and the most part, it will be fast because most of it should be cached. The installer will take care of this.
However, I added a property that lets the developer skip a conda environment re-creation If you want the conda env not to created you can use:
mvn clean compile -Dauto-rm-existing-python-env=
To skip the dependency install is more complex. There are a few ways of doing it. In the future, we can look at it as an enhancement. As of now, the dependencies are basically cached after the first time by pip in any case. It does not take very long.

start_server.sh works very slow for me. Previously server started slowly first time, but next starts were quick enough, but now all starts are too slow on my env.

I suspect that there is something going on with FastText that we can look into as a separate issue. Even when the cc.en.200.bin.gz model is not available. I see it being autodownloaded by FastText. This can be a separate ticket.
There is a different ticket that needs to boot up the ctxsever automatically when we start the NLPCraft server. I think that is a good time to look into performance issues and ways to optimize this.

second run - we check that models are already downloaded and do nothing with them - Also I guess python stuff shouldn't be reinstalled. What do you think ?

I think it should be reinstalled for now. If there is a change to the requirements.txt Python dependencies (as a result of code changes) the environment would be automatically adjusted.

if you want to re-download models for some reasons(broken installation etc) - find and delete the folder (which folder) and run mvn clean package again (like first run)

This will only be a delete of the models folder. We can include this in documentation later. I would suggest doing this AFTER we change the model downloads directory.
This is also a valid concern but I feel that this is out of the scope of this pull request as well.

Note please that we have some checks: https://github.com/apache/incubator-nlpcraft/pull/18/checks.
Now these tests failed.

conda is not installed there. I have sent an email about this to the dev group on
We have to support these tests. I guess we have to skip python files installations and all related tests based on some flag.
Something like this nlpcraft/pom.xml line 392
I could be mistaken but I don't think the python installations would make it fail. I think that these tests should pass if conda is available. This is the error that I have seen so far. Would this be possible to have conda on the git build? I'm not sure how to do this.


add Gendy!!

a repository for one existing Pd object https://github.com/ssfrr/gendyflext

this pd object is not "cannonical DSS", and has this concept of "target waveforms"
we can start with plain "cannonical" first 🙂

we should check code from SuperCollider and CSound **

perhaps code from Csound and SuperCollider is easier to steal 🙂 this Pd one uses lots of c++ and I don't like that
@timothyschoen

http://www.csounds.com/manual/html/gendy.html
http://gerard.paresys.free.fr/Theme/Theme7.html
https://composerprogrammer.com/research/stochasticsynthesis.pdf
source for supercollider
https://github.com/supercollider/supercollider/blob/7c481b571df6f4e1ba0fd5e0999751b63bad1224/server/plugins/GendynUGens.cpp


[improvement](disk balancer) reduce disk balance sensitivity

impr:

When disk usage < 1%,  it's very easy to trigger disk balance even disks usage diff is little. Fix this;
BE disk balance error do not print error code stack.

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Base table or view already exists: 1050 Table 'users' already exists

Laravel Version: 5.5.3
PHP Version: 7.1
Database Driver & Version: MariaDB 10.1.26

Description:
php artisan migrate
[Illuminate\Database\QueryException]
SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'users' already exists (SQL: create table users (id int unsigned not null aut
o_increment primary key, name varchar(255) not null, email varchar(255) not null, password varchar(255) not null, remember_token varchar
(100) null, created_at timestamp null, updated_at timestamp null) default character set utf8mb4 collate utf8mb4_unicode_ci engine = InnoDB R
OW_FORMAT=DYNAMIC)
[PDOException]
SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'users' already exists
Steps To Reproduce:
laravel new website
php artisan make:migration create_lists_table --create=lists
php artisan migrate
Problem
It Creates users table and give error but not creating lists table

there must be existing table 'users' already created, even before install

This repo is for bug tracking. Use the forums or slack channel for solving your issue

I have Solved My Problem Myself
by Changing My create_users_table.php
<?php

use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;

class CreateUsersTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::dropIfExists('users');
        Schema::create('users', function (Blueprint $table) {
            $table->increments('id');
            $table->string('name');
            $table->string('email')->unique();
            $table->string('password');
            $table->rememberToken();
            $table->timestamps();
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('users');
    }
}

Then close the issue

@niteshkumarniranjan thanks, your solution helped me too, viz, Schema::dropIfExists.

how can we migrate without losing out the data?

drop the database
create it again
php artisan migrate // will create all tables
php artisan make:migration create_test_table // create new migration file for test
php artisan migrate //check if works
it worked or me!
Good Luck

@benmi18  thank you so much you have saved my life!


Not able to connect tot the Wiki API from inside the web container.

I have been testing out Canasta to work on Kubernetes and found that the following line of code caused a curl timeout error when trying to reach the Wiki API from inside the web container. Which causes issues with Visual Editor.
Once the /etc/hosts inside the web container was edited and the line was removed, it worked perfectly.
https://github.com/CanastaWiki/Canasta/blob/master/_sources/scripts/run-apache.sh#L65
Since Kubernetes does not use the Docker gateway and the IP address used might be different for each instances on Kubernetes, it might be able to reach the Wiki API with the current /etc/hosts.
When I tested out, the Visual Editor was able to work without the modification of the /etc/hosts by the run-apache.sh. Is it necessary to connect to the API through Caddy in the case of Docker?

I think instead of just only define some new stuff on /etc/hosts, we could instead add a new entry on /etc/nsswitch.conf where it can pick up first hosts and then pick up the gateway if theres none.