kotovalexarian-likes-gitlab
/
gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity
gitlab-org--gitlab-foss/doc/development/README.md

178 lines
6.9 KiB
Markdown
Raw Normal View History

Exclude comments from specific docs
2017-11-01 15:56:40 +00:00
---
comments: false
Docs: add custom descriptions to most relevant docs
2018-05-09 11:07:11 +00:00
description: 'Learn how to contribute to GitLab.'
Exclude comments from specific docs
2017-11-01 15:56:40 +00:00
---
Make section headings consistent
2019-08-12 09:52:48 +00:00
# Contributor and Development Docs
Add doc file for dev rake tasks
2014-04-24 17:53:18 +00:00
Update docs to pass new markdownlint Deletes extra spaces and line, makes lists consistent, and fixes links.
2019-08-22 08:50:31 +00:00
## Get started
Structure the development documentation
2016-08-04 10:45:59 +00:00
Correct grammar (setup to set-up) in Docs
2018-09-19 16:03:00 +00:00
- Set up GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md)
Improve the contributing documentation This removes duplicated content from CONTRIBUTING.md and consolidate everything under doc/development/contributing/. This also fixes missing links. Lastly, this moves the style guides from the "Implement design & UI elements" page to a new "Style guides" page. Signed-off-by: Rémy Coutable <remy@rymai.me>
2018-10-11 11:57:49 +00:00
- [GitLab contributing guide](contributing/index.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Architecture](architecture.md) of GitLab
- [Rake tasks](rake_tasks.md) for development
Structure the development documentation
2016-08-04 10:45:59 +00:00
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
## Processes
Add latest changes from gitlab-org/gitlab@master
2019-09-18 14:02:45 +00:00
- [GitLab core team & GitLab Inc. contribution process](https://gitlab.com/gitlab-org/gitlab/blob/master/PROCESS.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Generate a changelog entry with `bin/changelog`](changelog.md)
Update list items to conform to documentation style guide
2019-06-17 03:54:09 +00:00
- [Code review guidelines](code_review.md) for reviewing code and having code reviewed
Document database review process See https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/6069
2019-07-19 17:33:48 +00:00
- [Database review guidelines](database_review.md) for reviewing database-related changes and complex SQL queries
Add latest changes from gitlab-org/gitlab@master
2019-09-17 14:16:34 +00:00
- [Pipelines for the GitLab project](pipelines.md)
Add "Guidelines for implementing Enterprise Edition features" in CE development doc since the doc page is already there Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-12-06 16:23:53 +00:00
- [Guidelines for implementing Enterprise Edition features](ee_features.md)
Add missing security process for developers
2018-05-08 10:55:02 +00:00
- [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer)
Add section to dev docs on accessing chatops closes 62127
2019-06-07 01:00:36 +00:00
- [Requesting access to Chatops on GitLab.com](chatops_on_gitlabcom.md#requesting-access) (for GitLabbers)
Add Danger bot development documentation This is overdue, and some of the suggestions are embarrassing, but this is how we actually develop Danger today.
2019-09-03 00:23:52 +00:00
- [Danger bot](dangerbot.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
Updates frontend to Frontend
2019-07-11 15:00:18 +00:00
## UX and Frontend guides
Structure the development documentation
2016-08-04 10:45:59 +00:00
deprecate UX Guide and redirect to design.gitlab
2018-11-21 15:25:15 +00:00
- [GitLab Design System](https://design.gitlab.com/) for building GitLab with existing CSS styles and elements
Creates Frontend Style guide
2017-03-22 19:30:54 +00:00
- [Frontend guidelines](fe_guide/index.md)
Add basic emoji development docs
2017-11-14 22:29:33 +00:00
- [Emoji guide](fe_guide/emojis.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
## Backend guides
Add documents for GitLab utilities
2017-12-29 06:29:53 +00:00
- [GitLab utilities](utilities.md)
Add latest changes from gitlab-org/gitlab@master
2019-10-07 06:06:10 +00:00
- [Issuable-like Rails models](issuable-like-models.md)
Add documentation on how to use structured logging
2018-11-13 22:43:16 +00:00
- [Logging](logging.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [API styleguide](api_styleguide.md) Use this styleguide if you are
Update list items to conform to documentation style guide
2019-06-17 03:54:09 +00:00
contributing to the API
Fix typo in doc/development/README.md
2018-06-12 03:28:29 +00:00
- [GraphQL API styleguide](api_graphql_styleguide.md) Use this
Fix minor grammar issue
2019-06-14 21:20:32 +00:00
styleguide if you are contributing to the [GraphQL API](../api/graphql/index.md)
Re-organize queues to use for Sidekiq Dumping too many jobs in the same queue (e.g. the "default" queue) is a dangerous setup. Jobs that take a long time to process can effectively block any other work from being performed given there are enough of these jobs. Furthermore it becomes harder to monitor the jobs as a single queue could contain jobs for different workers. In such a setup the only reliable way of getting counts per job is to iterate over all jobs in a queue, which is a rather time consuming process. By using separate queues for various workers we have better control over throughput, we can add weight to queues, and we can monitor queues better. Some workers still use the same queue whenever their work is related. For example, the various CI pipeline workers use the same "pipeline" queue. This commit includes a Rails migration that moves Sidekiq jobs from the old queues to the new ones. This migration also takes care of doing the inverse if ever needed. This does require downtime as otherwise new jobs could be scheduled in the old queues after this migration completes. This commit also includes an RSpec test that blacklists the use of the "default" queue and ensures cron workers use the "cronjob" queue. Fixes gitlab-org/gitlab-ce#23370
2016-10-21 16:13:41 +00:00
- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Working with Gitaly](gitaly.md)
- [Manage feature flags](feature_flags.md)
Docs: Merge EE doc/development to CE
2019-05-05 13:57:21 +00:00
- [Licensed feature availability](licensed_feature_availability.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [View sent emails or preview mailers](emails.md)
- [Shell commands](shell_commands.md) in the GitLab codebase
Document "No gems fetched from git repositories" policy [ci skip]
2017-01-27 11:17:21 +00:00
- [`Gemfile` guidelines](gemfile.md)
First version of pry_debugging.md
2018-07-20 05:31:51 +00:00
- [Pry debugging](pry_debugging.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Sidekiq debugging](sidekiq_debugging.md)
Document ways of accessing Session data in GitLab
2019-05-02 10:54:11 +00:00
- [Accessing session data](session.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Gotchas](gotchas.md) to avoid
Updates based on feedback
2017-11-21 15:15:24 +00:00
- [Avoid modules with instance variables](module_with_instance_variables.md) if possible
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [How to dump production data to staging](db_dump.md)
Rewrite the GitHub importer from scratch Prior to this MR there were two GitHub related importers: * Github::Import: the main importer used for GitHub projects * Gitlab::GithubImport: importer that's somewhat confusingly used for importing Gitea projects (apparently they have a compatible API) This MR renames the Gitea importer to Gitlab::LegacyGithubImport and introduces a new GitHub importer in the Gitlab::GithubImport namespace. This new GitHub importer uses Sidekiq for importing multiple resources in parallel, though it also has the ability to import data sequentially should this be necessary. The new code is spread across the following directories: * lib/gitlab/github_import: this directory contains most of the importer code such as the classes used for importing resources. * app/workers/gitlab/github_import: this directory contains the Sidekiq workers, most of which simply use the code from the directory above. * app/workers/concerns/gitlab/github_import: this directory provides a few modules that are included in every GitHub importer worker. == Stages The import work is divided into separate stages, with each stage importing a specific set of data. Stages will schedule the work that needs to be performed, followed by scheduling a job for the "AdvanceStageWorker" worker. This worker will periodically check if all work is completed and schedule the next stage if this is the case. If work is not yet completed this worker will reschedule itself. Using this approach we don't have to block threads by calling `sleep()`, as doing so for large projects could block the thread from doing any work for many hours. == Retrying Work Workers will reschedule themselves whenever necessary. For example, hitting the GitHub API's rate limit will result in jobs rescheduling themselves. These jobs are not processed until the rate limit has been reset. == User Lookups Part of the importing process involves looking up user details in the GitHub API so we can map them to GitLab users. The old importer used an in-memory cache, but this obviously doesn't work when the work is spread across different threads. The new importer uses a Redis cache and makes sure we only perform API/database calls if absolutely necessary. Frequently used keys are refreshed, and lookup misses are also cached; removing the need for performing API/database calls if we know we don't have the data we're looking for. == Performance & Models The new importer in various places uses raw INSERT statements (as generated by `Gitlab::Database.bulk_insert`) instead of using Rails models. This allows us to bypass any validations and callbacks, drastically reducing the number of SQL queries and Gitaly RPC calls necessary to import projects. To ensure the code produces valid data the corresponding tests check if the produced rows are valid according to the model validation rules.
2017-10-13 16:50:36 +00:00
- [Working with the GitHub importer](github_importer.md)
Add Import/Export dev docs
2019-01-14 18:18:54 +00:00
- [Import/Export development documentation](import_export.md)
Docs: Merge EE doc/development to CE
2019-05-05 13:57:21 +00:00
- [Elasticsearch integration docs](elasticsearch.md)
Add developer documentation on diff handling and limits
2018-04-09 21:24:26 +00:00
- [Working with Merge Request diffs](diffs.md)
Development guide for Kubernetes Start writing some development guidelines in relation to our Kubernetes integration
2019-01-23 20:48:38 +00:00
- [Kubernetes integration guidelines](kubernetes.md)
Document permissions for different entities
2018-09-10 10:54:52 +00:00
- [Permissions](permissions.md)
Add development documentation
2018-09-04 13:01:34 +00:00
- [Prometheus metrics](prometheus_metrics.md)
Document abstraction levels and code reuse This is based on the discussion in https://gitlab.com/gitlab-org/gitlab-ce/issues/49653 and the example merge request found at https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20908. The idea is fairly straightforward: we define a variety of abstractions, then state which can reuse which. This should ultimately foster _better_ code reuse, make it easier to debug problems, and make it harder to accidentally introduce a performance regression.
2018-08-16 15:14:14 +00:00
- [Guidelines for reusing abstractions](reusing_abstractions.md)
Link to DeclarativePolicy page from /development
2018-10-14 22:58:29 +00:00
- [DeclarativePolicy framework](policies.md)
Document how Git object deduplication works in GitLab
2019-03-19 13:25:12 +00:00
- [How Git object deduplication works in GitLab](git_object_deduplication.md)
Docs: Merge EE doc/development to CE
2019-05-05 13:57:21 +00:00
- [Geo development](geo.md)
Add developmenty documentation about routing Explain and recommend developers to use `/-/` scope for group and project routes in Rails Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2019-05-27 08:33:40 +00:00
- [Routing](routing.md)
Add Create Deep Dives to development docs See https://gitlab.com/gitlab-org/create-stage/issues/1 for more context
2019-06-18 11:11:58 +00:00
- [Repository mirroring](repository_mirroring.md)
- [Git LFS](lfs.md)
Document interacting GitLab components and features It's not uncommon that a single code change can reflect and interact with multiple parts of GitLab codebase, making it possible to break an existing feature. This commit expands documentation in that regard for developers.
2019-07-19 03:04:09 +00:00
- [Developing against interacting components or features](interacting_components.md)
Document uploads development guidelines Explain Workhorse acceleration features
2019-08-23 10:40:04 +00:00
- [File uploads](uploads.md)
Add latest changes from gitlab-org/gitlab@master
2019-10-02 06:06:28 +00:00
- [Auto DevOps development guide](auto_devops.md)
Structure the development documentation
2016-08-04 10:45:59 +00:00
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
## Performance guides
Structure the development documentation
2016-08-04 10:45:59 +00:00
Enhance some performance guideline titles This enhances a few performance guideline titles in the development README, making it easier to understand what topics they cover.
2018-09-18 12:25:22 +00:00
- [Instrumentation](instrumentation.md) for Ruby code running in production
environments
- [Performance guidelines](performance.md) for writing code, benchmarks, and
certain patterns to avoid
Added performance guidelines for new MRs These guidelines cover the performance requirement for newly submitted merge requests. These guidelines are put in to place to prevent merge requests from negatively impacting GitLab performance as much as possible.
2016-08-19 15:21:00 +00:00
- [Merge request performance guidelines](merge_request_performance_guidelines.md)
for ensuring merge requests do not negatively impact GitLab performance
Remove redundant word from development doc index
2018-09-18 20:17:07 +00:00
- [Profiling](profiling.md) a URL, measuring performance using Sherlock, or
tracking down N+1 queries using Bullet
Structure the development documentation
2016-08-04 10:45:59 +00:00
Add documentation about reading query plans This adds a database guide on how to read the output of "EXPLAIN" and "EXPLAIN ANALYZE", and how to use this output to understand a query's performance and optimise it.
2018-08-14 16:07:15 +00:00
## Database guides
### Tooling
- [Understanding EXPLAIN plans](understanding_explain_plans.md)
- [explain.depesz.com](https://explain.depesz.com/) for visualising the output
of `EXPLAIN`
Add pgFormatter as a database tooling Includes a PostgreSQL syntax beautifier as a database tooling.
2019-07-05 01:54:18 +00:00
- [pgFormatter](http://sqlformat.darold.net/) a PostgreSQL SQL syntax beautifier
Added documentation on adding database indexes
2016-08-11 12:22:21 +00:00
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
### Migrations
Added documentation on adding database indexes
2016-08-11 12:22:21 +00:00
Added guide about migrations and downtime This guide describes various commonly used operations and the impact they have on the availability of a GitLab instance.
2016-08-04 15:17:31 +00:00
- [What requires downtime?](what_requires_downtime.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [SQL guidelines](sql.md) for working with SQL queries
- [Migrations style guide](migration_style_guide.md) for creating safe SQL migrations
Add latest changes from gitlab-org/gitlab@master
2019-09-24 06:06:02 +00:00
- [Testing Rails migrations](testing_guide/testing_migrations_guide.md) guide
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Post deployment migrations](post_deployment_migrations.md)
- [Background migrations](background_migrations.md)
- [Swapping tables](swapping_tables.md)
Add latest changes from gitlab-org/gitlab@master
2019-10-14 15:06:07 +00:00
- [Deleting exiting migrations](deleting_migrations.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
### Best practices
- [Merge Request checklist](database_merge_request_checklist.md)
Added documentation on adding database indexes
2016-08-11 12:22:21 +00:00
- [Adding database indexes](adding_database_indexes.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
- [Foreign keys & associations](foreign_keys.md)
- [Single table inheritance](single_table_inheritance.md)
- [Polymorphic associations](polymorphic_associations.md)
- [Serializing data](serializing_data.md)
- [Hash indexes](hash_indexes.md)
- [Storing SHA1 hashes as binary](sha1_as_binary.md)
- [Iterating tables in batches](iterating_tables_in_batches.md)
- [Ordering table columns](ordering_table_columns.md)
- [Verifying database capabilities](verifying_database_capabilities.md)
Add basic docs for troubleshooting database problems
2017-11-16 03:19:14 +00:00
- [Database Debugging and Troubleshooting](database_debugging.md)
Track and act upon the number of executed queries This ensures that we have more visibility in the number of SQL queries that are executed in web requests. The current threshold is hardcoded to 100 as we will rarely (maybe once or twice) change it. In production and development we use Sentry if enabled, in the test environment we raise an error. This feature is also only enabled in production/staging when running on GitLab.com as it's not very useful to other users.
2018-01-15 15:21:04 +00:00
- [Query Count Limits](query_count_limits.md)
Add Gitlab::Database::Subquery.self_join to delete_all with limit `delete_all` doesn't support limit, so you'd need to subquery that. And instead of subquerying with `where(id: query)`, it's better to use an `INNER JOIN`. This method also works with MySQL, while subquerying doesn't (without another layer of subquerying) Reference: https://stackoverflow.com/questions/17892762/mysql-this-version-of-mysql-doesnt-yet-support-limit-in-all-any-some-subqu/17892886#17892886
2018-09-18 15:32:21 +00:00
- [Database helper modules](database_helpers.md)
Add code comments development guide Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2019-06-25 15:34:35 +00:00
- [Code comments](code_comments.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
Add latest changes from gitlab-org/gitlab@master
2019-10-14 15:06:07 +00:00
### Case studies
Add development documentation on label filtering This topic comes up periodically, and we've investigated several options for changing the way we filter labels. Unfortunately, we have yet to find an option that is strictly better than the current state.
2019-07-15 13:12:38 +00:00
- [Database case study: Filtering by label](filtering_by_label.md)
Adds new DB case study for namespaces statistics Includes a new database case study about all the approaches taken to store and update the namespace statistics in a performan manner performant approach to store and update the namespaces statistics
2019-08-22 12:12:28 +00:00
- [Database case study: Namespaces storage statistics](namespaces_storage_statistics.md)
Add development documentation on label filtering This topic comes up periodically, and we've investigated several options for changing the way we filter labels. Unfortunately, we have yet to find an option that is strictly better than the current state.
2019-07-15 13:12:38 +00:00
Docs: Realigning scattered EE docs into CE
2019-05-06 00:56:13 +00:00
## Integration guides
- [Jira Connect app](integrations/jira_connect.md)
Fix a missing link and create a separate "Testing guides" section Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-10-11 14:33:57 +00:00
## Testing guides
- [Testing standards and style guidelines](testing_guide/index.md)
- [Frontend testing standards and style guidelines](testing_guide/frontend_testing.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
## Documentation guides
Docs: add the documentation guidelines into its own dir
2018-06-06 11:32:38 +00:00
- [Writing documentation](documentation/index.md)
- [Documentation styleguide](documentation/styleguide.md)
Docs update documentation guidelines
2017-12-15 10:56:21 +00:00
- [Markdown](../user/markdown.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
## Internationalization (i18n) guides
Add guide to collaborate with i18n.
2017-05-16 18:26:12 +00:00
Move i18n/introduction to i18n/index
2017-10-09 09:28:01 +00:00
- [Introduction](i18n/index.md)
Docs: add translation and proof reading guidelines
2017-09-26 21:15:19 +00:00
- [Externalization](i18n/externalization.md)
- [Translation](i18n/translation.md)
Add guide to collaborate with i18n.
2017-05-16 18:26:12 +00:00
Add latest changes from gitlab-org/gitlab@master
2019-09-18 14:02:45 +00:00
## Event tracking guides
- [Introduction](event_tracking/index.md)
- [Frontend tracking guide](event_tracking/frontend.md)
- [Backend tracking guide](event_tracking/backend.md)
Refactor the Development documentation, and divide the Testing documentation into multiple pages
2017-10-11 05:59:34 +00:00
## Build guides
- [Building a package for testing purposes](build_test_package.md)
Structure the development documentation
2016-08-04 10:45:59 +00:00
## Compliance
- [Licensing](licensing.md) for ensuring license compliance
Add Go guidelines to developer docs
2019-02-06 16:10:12 +00:00
## Go guides
- [Go Guidelines](go_guide/index.md)
Add links to GDK autodevops and runner guides
2019-05-17 19:13:55 +00:00
Add shell scripting guides to development README
2019-08-05 14:47:46 +00:00
## Shell Scripting guides
- [Shell scripting standards and style guidelines](shell_scripting_guide/index.md)
Add latest changes from gitlab-org/gitlab@master
2019-09-25 18:06:17 +00:00
## Other Development guides
- [Defining relations between files using projections](projections.md)
Add links to GDK autodevops and runner guides
2019-05-17 19:13:55 +00:00
## Other GitLab Development Kit (GDK) guides
- [Run full Auto DevOps cycle in a GDK instance](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/auto_devops.md)
- [Using GitLab Runner with GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/runner.md)