openstack之kilo安装 对象存储服务

OpenStack对象存储是一个多租户的对象存储系统，它支持大规模扩展，可以以低成本来管理大型的非结构化数据，通过RESTful HTTP 应用程序接口。

它包含下列组件：

• Proxy servers (swift-proxy-server)
  接收OpenStack对象存储API和纯粹的HTTP请求以上传文件，更改元数据，以及创建容器。它可服务于在web浏览器下显示文件和容器列表。为了改进性能，代理服务可以使用可选的缓存，通常部署的是memcache。
• Account servers (swift-account-server)
  管理由对象存储定义的账户。
• Container servers (swift-container-server)
  管理容器或文件夹的映射，对象存储内部。
• Object servers (swift-object-server)
  在存储节点管理实际的对象，诸如文件。
• Various periodic processes
  为了驾驭大型数据存储的任务，复制服务需要在集群内确保一致性和可用性，其他定期进程有审计，更新和reaper。
• WSGI middleware
  掌控认证，使用OpenStack认证服务。

参考：Chapter 9. Add Object Storage

Install and configure the controller node

proxy service可以安装在任意节点，本文装在控制节点。代理服务依赖于例如认证服务所提供的认证和授权机制。但是，与其他服务不同的是，它也提供一个内部的机制可以在没有任何其它OpenStack服务的情况下工作。在配置对象存储服务之间，你必须生成服务凭证和API端点。

[Note]
对象存储服务在控制节点上不使用 SQL 数据库，使用在各个存储节点的分布式SQLite数据库。

获取admin权限:

#
source admin-openrc.sh

创建 swift user:

#
openstack user create swift --password swift

添加 admin role to the swift user:

#
openstack role add --project service --user swift admin

创建 swift service entity:

#
openstack service create --name swift \
  --description "OpenStack Object Storage" object-store

#
openstack service create --type  object-store --description "OpenStack Object Storage" swift

创建 Object Storage service API endpoint:

#
openstack endpoint create \
  --publicurl 'http://controller:8080/v1/AUTH_%(tenant_id)s' \  --internalurl 'http://controller:8080/v1/AUTH_%(tenant_id)s' \  --adminurl http://controller:8080 \  --region RegionOne \  object-store

安装软件包:

#
yum install -y openstack-swift-proxy python-swiftclient python-keystone-auth-token \
  python-keystonemiddleware memcached

从对象存储的仓库源中获取代理服务的配置文件：

#
curl -o /etc/swift/proxy-server.conf \  https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample?h=stable/kilo
  https:/

编辑 /etc/swift/proxy-server.conf 配置文件:

#
cp /etc/swift/proxy-server.conf /etc/swift/proxy-server.confbak
echo "[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift

[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo proxy-logging proxy-server[app:proxy-server]use = egg:swift#proxyaccount_autocreate = true[filter:keystoneauth]use = egg:swift#keystoneauthoperator_roles = admin,user# operator_roles = admin,__member__[filter:authtoken]paste.filter_factory = keystonemiddleware.auth_token:filter_factoryauth_uri = http://controller:5000auth_url = http://controller:35357auth_plugin = passwordproject_domain_id = defaultuser_domain_id = defaultproject_name = serviceusername = swiftpassword = swiftdelay_auth_decision = true[filter:cache]use = egg:swift#memcachememcache_servers = 127.0.0.1:11211[filter:healthcheck]use = egg:swift#healthcheck[filter:catch_errors]use = egg:swift#catch_errors[filter:ratelimit]use = egg:swift#ratelimit[filter:domain_remap]use = egg:swift#domain_remap[filter:cname_lookup]use = egg:swift#cname_lookup[filter:staticweb]use = egg:swift#staticweb[filter:tempurl]use = egg:swift#tempurl[filter:formpost]use = egg:swift#formpost[filter:name_check]use = egg:swift#name_check[filter:list-endpoints]use = egg:swift#list_endpoints[filter:proxy-logging]use = egg:swift#proxy_logging[filter:bulk]use = egg:swift#bulk[filter:slo]use = egg:swift#slo[filter:dlo]use = egg:swift#dlo[filter:container-quotas]use = egg:swift#container_quotas[filter:account-quotas]use = egg:swift#account_quotas[filter:gatekeeper]use = egg:swift#gatekeeper[filter:container_sync]use = egg:swift#container_sync[filter:xprofile]use = egg:swift#xprofile">/etc/swift/proxy-server.conf

[app:proxy-server]
use = egg:swift#proxy
account_autocreate = true

[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin,user
# operator_roles = admin,__member__
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = swift
password = swift
delay_auth_decision = true

[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:catch_errors]
use = egg:swift#catch_errors

[filter:ratelimit]
use = egg:swift#ratelimit

[filter:domain_remap]
use = egg:swift#domain_remap

[filter:cname_lookup]
use = egg:swift#cname_lookup

[filter:staticweb]
use = egg:swift#staticweb

[filter:tempurl]
use = egg:swift#tempurl

[filter:formpost]
use = egg:swift#formpost

[filter:name_check]
use = egg:swift#name_check

[filter:list-endpoints]
use = egg:swift#list_endpoints

[filter:proxy-logging]
use = egg:swift#proxy_logging

[filter:bulk]
use = egg:swift#bulk

[filter:slo]
use = egg:swift#slo

[filter:dlo]
use = egg:swift#dlo

[filter:container-quotas]
use = egg:swift#container_quotas

[filter:account-quotas]
use = egg:swift#account_quotas

[filter:gatekeeper]
use = egg:swift#gatekeeper

[filter:container_sync]
use = egg:swift#container_sync

[filter:xprofile]
use = egg:swift#xprofile">/etc/swift/proxy-server.conf

Install and configure the storage nodes

This section describes how to install and configure storage nodes that operate the account, container, and object services. For simplicity, this configuration references two storage nodes, each containing two empty local block storage devices. Each of the devices, /dev/sdb and /dev/sdc, must contain a suitable partition table with one partition occupying the entire device. Although the Object Storage service supports any file system with extended attributes (xattr), testing and benchmarking indicate the best performance and reliability on XFS. For more information on horizontally scaling your environment, see the Deployment Guide.

在您安装和配置卷服务之前，您必须先配置存储节点。类似于控制节点，存储节点包含一个管理网络接口上的网络。存储节点可以选择性地包含第二个网络接口到一个独立地网络上，用以复制。

配置所有节点 /etc/hosts 文件:

echo "# object1
10.0.0.51        object1

# object2
10.0.0.52        object2">> /etc/hosts

安装相关支撑软件包:

#
yum install -y xfsprogs rsync

格式化 /dev/sdb1 and /dev/sdc1 partitions as XFS:

#
fdisk /dev/sdb
fdisk /dev/sdc
mkfs.xfs /dev/sdb1
mkfs.xfs /dev/sdc1

创建 mount point directory structure:

#
mkdir -p /srv/node/sdb1
mkdir -p /srv/node/sdc1

编辑 /etc/fstab 文件:

#
echo "/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 2/dev/sdc1 /srv/node/sdc1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 2">>/etc/fstab
/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
/dev/sdc1 /srv/node/sdc1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 2"

挂载 devices:

#
mount /srv/node/sdb1
mount /srv/node/sdc1

编辑/etc/rsyncd.conf 文件:

#
echo "uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = MANAGEMENT_INTERFACE_IP_ADDRESS

[account]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/account.lock[container]max connections = 2path = /srv/node/read only = falselock file = /var/lock/container.lock[object]max connections = 2path = /srv/node/read only = falselock file = /var/lock/object.lock">/etc/rsyncd.conf

[container]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/container.lock

[object]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock">/etc/rsyncd.conf

[Note] Note
The rsync service requires no authentication, so consider running it on a private network.

启动 rsyncd service 并配置开机启动:

#
systemctl enable rsyncd.service
systemctl start rsyncd.service

[Note]
Perform these steps on each storage node.

安装软件包:

#
yum install -y openstack-swift-account openstack-swift-container \ openstack-swift-object
 openstack-swift-object

从对象存储资源仓库中获取帐户、容器和对象服务的配置文件：

# curl -o /etc/swift/account-server.conf \ https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample?h=stable/kilo
 https:/
# curl -o /etc/swift/container-server.conf \ https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample?h=stable/kilo
 https:/
# curl -o /etc/swift/object-server.conf \ https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample?h=stable/kilo
 https:/
# curl -o /etc/swift/container-reconciler.conf \ https://git.openstack.org/cgit/openstack/swift/plain/etc/container-reconciler.conf-sample?h=stable/kilo
 https:/
# curl -o /etc/swift/object-expirer.conf \ https://git.openstack.org/cgit/openstack/swift/plain/etc/object-expirer.conf-sample?h=stable/kilo
 https:/

编辑 /etc/swift/account-server.conf 文件:

#
cp /etc/swift/account-server.conf /etc/swift/account-server.confbak
echo "[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 6002
user = swift
swift_dir = /etc/swift
devices = /srv/node

[pipeline:main]
pipeline = healthcheck recon account-server

[app:account-server]
use = egg:swift#account

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

[account-replicator]

[account-auditor]

[account-reaper]

[filter:xprofile]
use = egg:swift#xprofile">/etc/swift/account-server.conf

编辑 /etc/swift/container-server.conf 文件:

#
cp /etc/swift/container-server.conf /etc/swift/container-server.confbak
echo "[DEFAULT]bind_ip = 0.0.0.0bind_port = 6001user = swiftswift_dir = /etc/swiftdevices = /srv/node[pipeline:main]pipeline = healthcheck recon container-server[app:container-server]use = egg:swift#container[filter:healthcheck]use = egg:swift#healthcheck[filter:recon]use = egg:swift#reconrecon_cache_path = /var/cache/swift[container-replicator][container-updater][container-auditor][container-sync][filter:xprofile]use = egg:swift#xprofile">/etc/swift/container-server.conf
bind_ip = 0.0.0.0
bind_port = 6001
user = swift
swift_dir = /etc/swift
devices = /srv/node

[pipeline:main]
pipeline = healthcheck recon container-server

[app:container-server]
use = egg:swift#container

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

[container-replicator]

[container-updater]

[container-auditor]

[container-sync]

[filter:xprofile]
use = egg:swift#xprofile"

编辑 /etc/swift/object-server.conf 文件:

#
cp /etc/swift/object-server.conf /etc/swift/object-server.confbak
echo "[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 6000
user = swift
swift_dir = /etc/swift
devices = /srv/node

[pipeline:main]
pipeline = healthcheck recon object-server

[app:object-server]
use = egg:swift#object

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock

[object-replicator]

[object-reconstructor]

[object-updater]

[object-auditor]

[filter:xprofile]
use = egg:swift#xprofile">/etc/swift/object-server.conf

编辑/etc/swift/container-reconciler.conf：

# cp /etc/swift/container-reconciler.conf /etc/swift/container-reconciler.confbak
echo "[DEFAULT]
# swift_dir = /etc/swift
# user = swift
# You can specify default log routing here if you want:
# log_name = swift

[container-reconciler]
# reclaim_age = 604800
# The cycle time of the daemon
# interval = 30
# Server errors from requests will be retried by default
# request_tries = 3

[pipeline:main]
pipeline = catch_errors proxy-logging cache proxy-server

[app:proxy-server]
use = egg:swift#proxy

[filter:cache]
use = egg:swift#memcache

[filter:proxy-logging]
use = egg:swift#proxy_logging

[filter:catch_errors]
use = egg:swift#catch_errors">/etc/swift/container-reconciler.conf

编辑/etc/swift/object-expirer.conf：

#
cp /etc/swift/object-expirer.conf /etc/swift/object-expirer.confbak
echo "[DEFAULT]
# swift_dir = /etc/swift
# user = swift
# You can specify default log routing here if you want:
# log_name = swift
# log_facility = LOG_LOCAL0
# log_level = INFO
# log_address = /dev/log
# The following caps the length of log lines to the value given; no limit if
# set to 0, the default.
# log_max_line_length = 0

[object-expirer]
# interval = 300
# auto_create_account_prefix = .
# expiring_objects_account_name = expiring_objects
# report_interval = 300
# concurrency = 1

[pipeline:main]
pipeline = catch_errors proxy-logging cache proxy-server

[app:proxy-server]
use = egg:swift#proxy
# See proxy-server.conf-sample for options

[filter:cache]
use = egg:swift#memcache
# See proxy-server.conf-sample for options

[filter:catch_errors]
use = egg:swift#catch_errors
# See proxy-server.conf-sample for options

[filter:proxy-logging]
use = egg:swift#proxy_logging">/etc/swift/object-expirer.conf

确认挂载点目录结构是否有合适的权限：

#
chown -R swift:swift /srv/node

创建 recon目录并确认它有合适的权限：

#
mkdir -p /var/cache/swift
chown -R swift:swift /var/cache/swift

Create initial rings

帐户服务器使用帐户 ring 来维护一个容器的列表。

[Note]
Perform these steps on the controller node.

切换到/etc/swift目录。

#
cd  /etc/swift

创建基本的account.builder文件：

#
swift-ring-builder account.builder create 10 3 1

[Note]
This command provides no output.

Add each storage node to the ring:

在每个存储节点执行:

#
swift-ring-builder account.builder add r1z1-10.0.0.51:6002/sdb1 100
swift-ring-builder account.builder add r1z2-10.0.0.51:6002/sdc1 100
swift-ring-builder account.builder add r1z3-10.0.0.52:6002/sdb1 100
swift-ring-builder account.builder add r1z4-10.0.0.52:6002/sdc1 100

验证 ring contents:

#
swift-ring-builder account.builder

Rebalance the ring:

#
swift-ring-builder account.builder rebalance

Container ring

对象服务器使用对象环来维护对象在本地设备上的位置列表。

To create the ring

[Note]
Perform these steps on the controller node.

切换到 /etc/swift 目录.

#
cd /etc/swift

创建 base container.builder file:

#
swift-ring-builder container.builder create 10 3 1

[Note] Note
This command provides no output.

添加每个 storage node to the ring:

#
swift-ring-builder container.builder add r1z1-10.0.0.51:6001/sdb1 100
swift-ring-builder container.builder add r1z2-10.0.0.51:6001/sdc1 100
swift-ring-builder container.builder add r1z3-10.0.0.52:6001/sdb1 100
swift-ring-builder container.builder add r1z4-10.0.0.52:6001/sdc1 100

验证 ring contents:

#
swift-ring-builder container.builder

Rebalance the ring:

#
swift-ring-builder container.builder rebalance

Object ring

The object server uses the object ring to maintain lists of object locations on local devices.

To create the ring

[Note] Note
Perform these steps on the controller node.

切换到 /etc/swift 目录.

#
cd /etc/swift

创建 base object.builder file:

#
swift-ring-builder object.builder create 10 3 1

[Note] Note
This command provides no output.

添加每个 storage node to the ring:

#
swift-ring-builder object.builder add r1z1-10.0.0.51:6000/sdb1 100
swift-ring-builder object.builder add r1z2-10.0.0.51:6000/sdc1 100
swift-ring-builder object.builder add r1z3-10.0.0.52:6000/sdb1 100
swift-ring-builder object.builder add r1z4-10.0.0.52:6000/sdc1 100

验证 ring contents:

#
swift-ring-builder object.builder

Rebalance the ring:

#
swift-ring-builder object.builder rebalance

Distribute ring configuration files

复制 account.ring.gz、container.ring.gz和 object.ring.gz文件到每个存储节点和其他运行了代理服务的额外节点的 /etc/swift目录下：

#
scp *.ring.gz object1:/etc/swift
scp *.ring.gz object2:/etc/swift

Finalize installation

配置哈希和默认的存储策略

从对象存储的仓库源中获取 /etc/swift/swift.conf文件：

#
curl -o /etc/swift/swift.conf \  https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample?h=stable/kilo
  https:/

编辑 /etc/swift/swift.conf 文件:

#
echo "[swift-hash]
swift_hash_path_suffix = HASH_PATH_PREFIX
swift_hash_path_prefix = HASH_PATH_SUFFIX
[storage-policy:0]
name = Policy-0
default = yes
[swift-constraints]">/etc/swift/swift.conf

复制 swift.conf文件到每个存储节点和其他运行了代理服务的额外节点的 /etc/swift目录下：

#
scp /etc/swift/swift.conf object1:/etc/swift
scp /etc/swift/swift.conf object2:/etc/swift

在所有节点上，确认配置文件目录是否有合适的所有权：

#
chown -R swift:swift /etc/swift

在控制节点和其他运行了代理服务的节点上，启动对象存储代理服务及其依赖服务，并将它们配置为随系统启动：

#
systemctl enable openstack-swift-proxy.service memcached.service
systemctl start openstack-swift-proxy.service memcached.service

在存储节点上，启动对象存储服务，并将其设置为随系统启动：

#
systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service \
  openstack-swift-account-reaper.service openstack-swift-account-replicator.service
systemctl start openstack-swift-account.service openstack-swift-account-auditor.service \  openstack-swift-account-reaper.service openstack-swift-account-replicator.servicesystemctl enable openstack-swift-container.service openstack-swift-container-auditor.service \  openstack-swift-container-replicator.service openstack-swift-container-updater.servicesystemctl start openstack-swift-container.service openstack-swift-container-auditor.service \  openstack-swift-container-replicator.service openstack-swift-container-updater.servicesystemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \  openstack-swift-object-replicator.service openstack-swift-object-updater.servicesystemctl start openstack-swift-object.service openstack-swift-object-auditor.service \  openstack-swift-object-replicator.service openstack-swift-object-updater.service
  openstack-swift-account-reaper.service openstack-swift-account-replicator.service
systemctl enable openstack-swift-container.service openstack-swift-container-auditor.service \
  openstack-swift-container-replicator.service openstack-swift-container-updater.service
systemctl start openstack-swift-container.service openstack-swift-container-auditor.service \
  openstack-swift-container-replicator.service openstack-swift-container-updater.service
systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \
  openstack-swift-object-replicator.service openstack-swift-object-updater.service
systemctl start openstack-swift-object.service openstack-swift-object-auditor.service \
  openstack-swift-object-replicator.service openstack-swift-object-updater.service

Verify operation

[Note] Note
The swift client requires the -V 3 parameter to use the Identity version 3 API.

[Note] Note
Perform these steps on the controller node.

Source the demo credentials:

#
source admin-openrc.sh

显示服务状态：

#
swift -V 3 stat

上传一个测试文件：

#
touch FILE
swift -V 3 upload demo-container1 FILE

Replace FILE with the name of a local file to upload to the demo-container1 container.

显示 containers:

#
swift -V 3 list

下载 test file:

#
swift -V 3 download demo-container1 FILE